One of APAC's leading investment management is seeking talented data engineer with strong web-scraping experience to drive projects crucial to the decision-making of their investments. You will be responsible for extracting and ingesting data from websites using web crawling tools, owning the creation process of these tools, services, and work flows to improve crawl /scrape analysis.
- Be hands-on in the design and development of large-scale data extraion and ingestion pipelines from different sources
- Provide technical solutions to web scraping and APP scraping projects
- Help team on data warehousing and ETL
- Help team resolve any issue related to third-party datasets
- At least 5 years of direct exposure to web scraping or related
- A track record of running large scale web scrapes, enterprise-level web crawlers and data pipelines
- Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, pandas, SQL, BeautifulSoup, etc)
- Experience with system monitoring/administration tools
- Good understanding of HTTP, TCP and other network protocols
- Cloud-based data management experience
- Ideally experience of game development or online gaming platform
- Fluency in English and/or Mandarin Chinese
Overseas applicants are also welcome. Feel free to email [email protected] or WhatsApp +852 6290 7292.