Job Responsibilities
Develop, test, maintain and operate a large scale distributed multithreaded web crawler in a Linux environment.

Qualifications

• 5+ years of Java or C/C++ programming experience
• 2+ years of sockets programming experience
• 2+ years of threaded programming experience
• Expertise and thorough understanding of protocols related to web crawling, including: TCP/IP, HTTP, HTML, Robot Exclusion, cookies, Javascript
• Must have extensive development experience with a scripting language: Perl, Python, Tcl/sh or PHP.
• Substantial experience programming in the Linux/Unix environment
• Knowledge of mySQL or other database server is a plus
• Passion and interest in building consumer applications
  

Apply Now

Back