Crawl Domain Find Expred Domains

$10-30 USD

Chiuso

Pubblicato

quasi 5 anni fa

$10-30 USD

Pagato al completamento

We are looking for a crawler to crawl every page of a website looking for external links pointing to expired domains. User should definde a list of sites to crawl via text file. Crawler should work logically crawling all pages of a site and not be sitemap dependent. Only unique external domains should be logged to prevent duplicate domain availability lookups. User should also be able to define a list of urls to ignore checking for availability; eg. [login to view URL] etc. these domains should be user defined in a blacklist text file. Results should be given in a csv file listing linking domain and available domain.

Python

Web Scraping

Rif. progetto: 19760507

Info sul progetto

4 proposte

Progetto a distanza

Attivo 5 anni fa

Hai voglia di guadagnare un po'?

Indirizzo Email

I vantaggi delle offerte su Freelancer

Imposta il tuo budget e le scadenze

Fatti pagare per il lavoro svolto

Delinea la tua proposta

La registrazione e le offerte sui lavori sono gratuite

4 freelance hanno fatto un'offerta media di $74 USD

@chirgeo

Hi. I did read the project description and have a few questions. 1. Do you need the script as well or data only? 2. What is the format of the output data? CSV is OK? We can do other formats as well. 3. Which fields do you want to extract from the website? Thx, waiting for these details and hope to collaborate.

$200 USD in 5 giorni

5,0

(155 valutazioni)

8,1

@smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHON (Scrapy, Selenium) based web scraper as well as WINDOWS BASED web scraping software through which I have crawled many sites such as Craigslist, Amazon, Yelp and many others. I have also worked on complex site to bypass CAPTCHA with the use of PROXY IP bouncing techniques.. Let's work together :) Have a great day! I am glad to see your WORK HISTORY and positive reviews of other freelancers. I am really excited to work with you and would love to have a long-term business association for any of your data related needs less ,,,,,,,,,,,,. ,

$35 USD in 3 giorni

4,9

(120 valutazioni)

6,1

@arundevp

Hey, I think i may be able to help you out on this project since i have worked on a similar project at work. Im new to freelancer, though i have years of experience in software development in python.

$20 USD in 1 giorno