Develope small scraper / crawler

$10-30 USD

Completato

Pubblicato

più di 8 anni fa

$10-30 USD

Pagato al completamento

I am looking for a guy who can realize the following project: In Google, the following search string is inserted: "[search keyword] site:*/impressum" What then shall be done is the following: - The urls of all findings are extracted - The urls shall be saved in a DB - The urls are cut that way that just the domain name with the tld (with http://www. in front) is saved in the DB Example: - finding is [login to view URL] - saved in DB: [login to view URL] (if the domain is already in DB, it is not saved twice) - I will provide a list of proxies to be used for that project - The project is done when a test with a certain keyword (32,900 matches) was successful

MySQL

PHP

Python

Rif. progetto: 8990966

Info sul progetto

9 proposte

Progetto a distanza

Attivo 8 anni fa

Hai voglia di guadagnare un po'?

Indirizzo Email

I vantaggi delle offerte su Freelancer

Imposta il tuo budget e le scadenze

Fatti pagare per il lavoro svolto

Delinea la tua proposta

La registrazione e le offerte sui lavori sono gratuite

Assegnato a:

@czrain

I'm an experienced Python developer with 10+ software engineering experience. Worked for a couple years at Google, amongst others. For this, I would probably go with Selenium browser automation, for one very particular reason: Google tends to not like web automation using eg. crawlers and puts script obstacles in the way, whereas a real (or emulated browser) will do just fine. Otherwise, this seems like a decently simple task, the only consideration is that google gives you results gradually so browsing through all the pages will take a while. (Especially if there's 32900 matches!) Also, acceptance criteria is weird, you're probably aware that the number of results google gives you is an estimate and there isn't an actual 32900 individual links.

$50 USD in 1 giorno

4,9

(4 valutazioni)

3,0

9 freelance hanno fatto un'offerta media di $81 USD

@sknadeem123

hi i am ready to provide this type application for easy to use in which technology you wish to use ? Java or Web apps ?etc ?

$111 USD in 4 giorni

5,0

(17 valutazioni)

4,5

@polaris0621

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

$157 USD in 1 giorno

5,0

(8 valutazioni)

4,5

@deescorp

I already have a product like that. the application is build on C# and takes input of search query in files format. Then search the google,bing, yahoo and save the results in database of mysql. I can give you the product so that you can use it on your own. Let me know and i will modify that according to your needs. Thanks

$55 USD in 3 giorni

4,9

(10 valutazioni)

3,9

@owenzhu

expirenced scraper, has crawled itunes app store, twitter, sina weibo, etc, good at database as well

$55 USD in 3 giorni

5,0

(8 valutazioni)

3,8

@marketiplace

I got 7+years work experience in Data Collection,Bulk Email Campaign,Excel VBA and Internet Research in IT companies here.I can do create script and scrap datas from site Directory and yellowpages using C++,Python and Perl coding as per your requirements in excel with multiple ip rotations.I have dealt with US,UK and Australia companies President,Directors and Managers for web design and development projects successfully and I have Good Communication with writing skills.I am well versed in Internet,MS Office Applications and Phone Etiquette manners with latest Technologies.I can accept your payment terms.

$55 USD in 1 giorno