About the Role
Description:
Seeking a talented and experienced Sr. Python Developer. The ideal candidate will have a deep understanding of Python Scrapy, including experience with crawling, proxy management, unit testing, data extraction, and database management.
1) Develop and maintain efficient and scalable Python code to support web crawling and data extraction.
2) Implement proxy management systems to ensure secure and reliable web crawling.
3) Create and execute unit tests to ensure the quality and stability of the code.
4) Design and implement data engineering solutions to process and analyze large volumes of data.
5) Work with databases to create and manage data tables and queries.
6) Collaborate with other team members to ensure successful project delivery.
Requirements
Requirements:
Strong experience with Scrapy, unit testing frameworks, and database technologies.
Experience with proxy management and distributed systems is a plus.
Familiarity with big data technologies such as Hadoop, Spark, and AWS is a plus.
Excellent problem-solving skills and ability to work independently or as part of a team. Strong communication and interpersonal skills.
About the Company
Aggregate Intelligence is a platform-agnostic fully integrated BI provider in vertical markets such as travel, retail, real estate and more. Our focus is on the seamless delivery of data intelligence to the user – whether that is in raw data feed form via APIs, in dynamic spreadsheet/platform integrations or in custom-build vertical BI solutions targeting a specific problem for a specific user base.