Data Automation and Curation

Hey guys, nice to have you back here on my page. Hope everyone has been doing great. Get excited, I am going to tell you about the project I am currently working on for the Outreachy season.
Okay so let's get right to it

My project for the season is themed: "

Automated versioning and curation of land sector datasets

". Very exciting right? Yes. So...

What is automated versioning and curation?
Automated versioning refers to the process of automatically generating and managing version numbers for software or code repositories based on predefined rules or triggers. This practice streamlines the tracking of changes and updates in a systematic manner, enabling developers to easily identify and differentiate between different iterations of the software, ensuring better collaboration, traceability, and compatibility management throughout the development lifecycle.

Data curation refers to the systematic process of collecting, organizing, validating, and maintaining digital information to ensure its quality, relevance, and accessibility over time. This involves tasks such as data collection, cleaning, transformation, metadata creation, and documentation, all aimed at preserving and enhancing the value of data for analysis, interpretation, and sharing within various domains, such as research, business, and cultural heritage, while also addressing issues of accuracy, consistency, and long-term usability.

Okay enough with the terms, let me tell you about this awesome organization I am learning a lot from.

Moja global is a collaborative project under the Linux Foundation that supports ambitious climate action by bringing together a community of experts to develop open-source software – including the groundbreaking FLINT software – which allows users to accurately and affordably estimate greenhouse gas emissions and removals from forestry, agriculture and other land uses (AFOLU).

Anyone(I mean literally anyone(including you especially)) can make a contribution to the software, the science, the documentation, and the promotion of Moja global’s tools.

Moja global's mission is to make these datasets flint ready and tracked to ensure that data analyst and scientist all over the world can always see the changes that the dataset has undergone. So basically this datasets are fetched from the original sources using python pipeline scripts and processed using the gdal software.... the processed data is then tracked using dvc. this ensures that all the changes to the file are noted.... this all happens using GHA and the final tracked files are pushed to gdrive... Moja global ensures that the data is first make flint ready with an accompanying notebook or file description to tell new contributors how to go around understanding the work behind the files....

Great communication skills in writing, project documentation and data management were very valuable to participate in this project. I was working as part of a team to document the project. I am also develop expertise in data analysis and team collaboration. Skills in Python, git, dvc, R or C++ are very desirable for this project. And so you should know any code developed under this project is open source to facilitate GHG models by experts around the world.

How Far Can You Go

Welcome to week 7 of the Outreachy Internship Program.  Its been a while since we last connected, I'm sure you must be wondering what I have been up to. Well I have been up to a lot and I am here to tell you all about it. This project is really something.

Just a reminder, the project that I am working on is: "Automated versioning and curation of land sector datasets". The original timeline for the project half looks something like this. You can feel that's quite a lot to do in three months right? lol, not really. 

The trick is once you start working, time doesn't matter anymore. Every task you carryout becomes seamless.





You see an open source project is usually huge because you have a wide array of experts from around the globe contributing and maintaining the project. Sometimes it can be quite hectic to know exactly when and how to fit in. This is why we have a community that guides and supports newcomers. Specifically, In the outreachy internship, we are assigned project supervisors that support us whenever and wherever we face any problem.  I have the best mentors on this project, they are the real deal.

So when you first join Moja global, the fist thing to do is to understand the science behind the part of the project you will be contributing to. I am contributing to the FLINT Reporting tool. The FLINT Reporting Tool provides Business Intelligence for analyzing and transforming FLINT output databases into useful information and outputs.

Sometimes, it can be difficult to attain the goals we set. Sometimes we need to revise the goals regularly so that we can attain the overall objective. The secret to success is in Consistency and Persistency. You just have to wiling to work hard to attain the main objective.

Currently, we have attained three set goals on our project timeline. This week we are moving to the fourth. Honestly some of the attained goals proved to be a real challenge. But with the massive support of my mentors, I got through with it. If you are working on an objective and it is taking longer than expected, please understand that in coding, it is completely normal, it happens.

Imagine at this time, I had to start the project all over. Evidently, I will do a lot of  things differently. I understand how to actively join an open source project. I know i a have to understand the science and philosophy of these projects to make valuable contributions. It would be a whole new adventure because i would do it differently.

In the second half of this project, I really desire to attain the overall objective. Moreover, the organization is really a social comfort zone and the projects are environmental friendly. I mean Climate Action should be taken seriously. Together we can do a lot to mitigate and adapt to Climate change through data automation and curation of the datasets.