Data Engineer – Tech Solutions Ltd (Jan 2022 – Present)
Automated web extraction to collect structured product pricing datasets from multiple e-commerce sources. The work involved building repeatable pipelines that gather, standardize, and deliver data suitable for downstream analysis or machine learning. This activity functions as data collection for training/analytics rather than manual annotation. • Built scraping pipelines to collect pricing data from 50+ e-commerce sites. • Used Scrapy and Selenium to automate extraction workflows and reduce manual effort. • Created Power BI dashboards to visualize collected datasets for business stakeholders. • Maintained data ingestion pipelines to keep extracted datasets consistent over time.