Details

Olcademy is hiring for the position of Data Engineering Intern!

Responsibilities

• Web Scraping: Design, develop, and deploy web scraping programs using Python and libraries like Scrapy or BeautifulSoup.
• Data Extraction: Implement efficient techniques to handle diverse website structures and content formats.
• Data Cleaning and Transformation: Ensure the quality and consistency of scraped data through robust cleaning and transformation processes.
• Pipeline Development: Build and maintain scalable scraping pipelines to support large-scale data collection.
• Monitoring and Troubleshooting: Identify and resolve issues such as website changes or access restrictions during scraping operations.
• Collaboration: Work closely with data analysts and scientists to ensure the scraped data aligns with their requirements.
• Research and Ethics: Stay updated on the latest web scraping methods while adhering to ethical standards.
• Documentation: Write clear documentation for your code and processes to enhance maintainability and knowledge sharing.
• Machine Learning Integration: Understand and incorporate machine learning algorithms where applicable.
• Data Transformation: Convert raw, complex data into structured and consumable formats.
• Agile Methodology: Work effectively within an Agile framework.

Requirements

• Educational Background: Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent experience).
• Technical Expertise:
  • Proven experience developing web scraping programs and scripts.
  • Proficiency in Python and its web scraping libraries (e.g., Scrapy, BeautifulSoup).
  • Experience with data extraction techniques such as regular expressions and XPath.
  • Familiarity with tools for data cleaning and transformation (e.g., pandas, Spark) (preferred).
  • Basic understanding of web technologies (HTML, CSS, JavaScript) (preferred).

This is a great opportunity to gain hands-on experience in data engineering, working on real-world projects while contributing to innovative solutions at Olcademy!