sqly-me

Project Info

Porretas thumbnail

Project Description


Read the README description at the repository to better understand how it works.

However, a simple description is: When there is a need of using more than one dataset, there is a need to cross them at some point. This project is a Python Module capable of doing such merge automatically. It is an enabling tool to allow data scientists (for instance) to avoid the effort of doing it. When it merges all the structured data the user wants, it returns a report and a sqlite database file for the user to then proceed on their analysis.

This project proposes a solution for multiple challenges by creating a Python module capable of taking multiple .csv or .xslx files and combining them into one single database, on a sqlite file. In addition, it provides a report with all files that were able to merge into such database.
It is an enabling tool, not a data analysis and information giving software. However, it is capable of checking all possible combinations inside the total range of files given. For instance, if 10 files are given and two different clusters of data exists on them, it will generate 2 different databases alongside with their respective reports.


Data Story


All outcomes from the data used on this project is present at the Google Drive folder:
https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws?usp=sharing


Evidence of Work

Video

Homepage

High-Res Image

Team DataSets

Digital Culture

Description of Use: For this challenge, multiple datasets from this URL were used. Which ones were used and which ones successfully merged are presented at the report on the Google Drive Folder: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws

Data Set

Physical Culture

Description of Use: For this challenge, multiple datasets from this URL were used. Which ones were used and which ones successfully merged are presented at the report on the Google Drive Folder: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws

Data Set

Households in Housing Stress

Description of Use: Investing in Regions datasets used. Showcasing our Regions datasets. In addition to this one, these following datasets were used: https://data.sa.gov.au/data/dataset/households-in-25-housing-stress https://data.sa.gov.au/data/dataset/housing-stress-50-of-income The outcome is displayed at the team google drive folder: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws

Data Set

TM-Link Data Discovery

Description of Use: We crossed both TM-Link's application and applicants dataset to get an overview of the given data. The outcome is a report giving an analysis of that merge. However, the real outcome on the project is the capability of crossing any other dataset, as the user prefers. The report is found on the following link: https://drive.google.com/drive/folders/13G5ToX0MXUi_SUyzVlTOr6DN-btfo4ws?usp=sharing

Data Set

Small Area Labour Markets (SALM) Data

Description of Use: Small Area Labour Markets (SALM) Data redirects to the following page: https://employment.gov.au/small-area-labour-markets-publication From that page, two different .csv files (SA2 and LGA data tables) were downloaded and used on the project. The outcome is a report and a sqlite database file containing the united data of those and provided on the Google Drive folder, on the team details

Data Set

Challenges

Investing in Regions

Region: South Australia

Challenge

Showcasing our regions

Region: South Australia

Challenge

Thrive or survive: how can we adapt for the future?

Region: Australia

Challenge

Digital Culture

Region: South Australia

Challenge

Physical Culture

Region: South Australia

Challenge

Australia@Sea: what is our future relationship with the ocean environment?

Region: Australia

Challenge

Australia’s Future Employment

Region: Australia

Challenge

TM-Link Data Discovery

Region: Australia

Challenge

Local Government Information Technology Association of South Australia

Region: South Australia

Challenge

Helping a social impact ‘start up’ (small organisation) to tell their story

Region: Australia

Challenge

Back to Projects