We look for scraping expert solving our requirements in Java.
Goal is to have later offline browsable scraped content of a section of websites.
Your job will be to implement the crawler to scrape based on a URL regex and to scrape the visited pages into a folder per page.
after the first run, the scraper shall check for changes on the page and persist the changed pages in new folders.
Typically each website-domain contains about 20 subpages, which are relevant to us.
Depending on the choosen path, you will require a specific solution like selenium, jsoup and so on.
Import is, that the content is offline browsable incl. the images!
Milestones
MS1:
Implement a simple crawler on a shared page which is downloading the sites for offline use.
It creates also screenshots of the visited pages, so that if something goes wrong the image view is at least available.
MS2:
- implement a spring boot cron job, which runs your code on scheduled basis
MS3:
- implement the difference building and persisting if found differences
MS4:
- fix and enhance the scraper for more shared pages
Implementations:
- the impl interface for the next named service classes => share for approval
- service class(es), which solve the scraping, screenshoting, persisting, comparing
NO UI for now required!
NO database required!
No REST endpoint exposing for the above named methods required! (only consuming the apis in the given links)
Your background is:
- multiple years of experience with Java
- multiple years of experience with web scraping
- multiple years of experience with spring boot
If you are a good fit, you are open to get more tasks about implementing solutions fully on your own (e.g. with your team)
Budget?
will not be disclosed, place your best bid to get considered
What is next?
We will share you a NDA and afterwards a paid test task.
Payment?
- you estimate in a WBS (optimistic, expected, pessimistic, where optimistic < expected < pessimistic) after getting the task
- we discuss about clearances and effort
- we mutually agree to effort
- we assign you the task after mutually agreed
- you implement & delivery
- we pay
(basically the rules of freelancer)
Closed book vs open book?
We work only on open book.
Closed book means you are unwilling to define a WBS for the work and you add only a price tag to the task.
We are sorry we will not hire you in such a case!
Deliveries?
- in our on premise git (access will be granted to you)
- full sources
- maven
- libs, need prior confirm and we prefer to use mostly latest stable versions
- JDK 17 (mostly LTS)
- CI/CD works fine with checkstyle, pmd, spotbugs
- works on our side too (stage + production)
Hello there,
I am George, and I've been a web crawler expert for multiple years. I'm confident that my experience fits the criteria of your Java scraping expert job. My expertise aligns with the challenge you mention-scraping a section of websites into an offline browsable format along with images and setting up cron jobs.
I recently completed a job which requires a similar custom web scraping tool, giving me the experience needed to handle your job.
1) What is the exact timeline for the job?
2) How structured and organised is the provided data?
3) Do you have any specific requirements regarding libraries, frameworks or coding practices?
4) What methodologies will you use to check the quality of my work?
5) Are there any other details that I need to be aware of?
I look forward to hearing from you and discussing further possibilities.
Best regards,
George
Hello!
I'm a Java scraping specialist, capable of meeting your unique data extraction needs. I'll design a custom solution to efficiently scrape and deliver the data you require while maintaining data integrity. Let's discuss your specific requirements and optimize your data retrieval for your project's success.
Thank you-
Dear Sir,
I'm an expert in web automation, web scraping and lead generation using the latest technologies like Java, Jsoup, selenium, python etc. Please ping me I can discuss your project in more detail and can start immediately.
Looking forward!!
Thanks & Regards
Hello, my name is Sandeep, and I am the founder of ESOLUTIONS SS. We understand that you are looking for someone to implement a Java scraping expert solution for your requirements in Java. Specifically, you are looking for later offline browsable scraped content of a section of websites. Your goal is to have the scraped content later available offline for use.
ESOLUTIONS SS has developed multiple solutions that can meet your needs such as selenium, jsoup, etc. We have experienced freelancers with multiple years of experience in web scraping who can assist you with this project. Additionally, our services include Google API access and Xero services which makes them ideal for this project.
⭐⭐⭐⭐⭐
Hi there,
I have strong expertise in web crawler, web scraper, web monitor, web automation and b-o-t-s.
I have already made lots of scrapers and b-o-t-s for sites such as Google, Google Map, Amazon, Twitter, Instagram, Facebook, eBay and many more with dynamic proxy support and multi threading 24/7 support for running on VPS.
I have read the details and looks an easy job for me.
Have a look at my profile for my portfolio and similar projects I've completed with reviews.
Message me, I would like to discuss further details over chat.
Hello, I'm an expert Java developer with experience in object-oriented, data structure, swing, JavaFX and more advanced. I can help you to finish this project with great quality.
We can negotiate on price/Budget Regards
Relevant Skills and Experience java.
Hi. Experienced web scraping developer here with 8+ years of hands-on experience, I can assist you in building a Java-based scraper to achieve your offline content scraping goals.
To proceed further, I kindly request the following information:
1. Can you provide the specific websites or URL patterns that you want to scrape content from?
2. Are there any particular features or functionalities you expect from the scraper?
3. What is the expected volume of data to be scraped and stored offline?
4. Do you have any preferences for the tools or libraries like Selenium or Jsoup?
Please initiate a chat so we can discuss the project thoroughly. I look forward to collaborating with you to create an efficient web scraper for your needs.
We are looking for an experienced Java scraper expert to help us with our requirements in Java. Specifically, we are looking for someone who can implement a crawler on a shared page that is downloading the sites for offline use so that screenshots can be taken of the visited pages if anything goes wrong.
Hi
I'm a skilled software engineer, and I'm a hard worker,
I've seen your job description (( Java Scraping Expert )) and I'm ready to help you skillfully.
Hope to hear from you soon.
Best regards,
Said.
Hello there! My name is vishva and I'm a member of the Dqubex team. We specialize in Java, web scraping, spring boot and more. I understand that you are looking for an expert to solve your requirements in Java and have later offline browsable scraped content. With our experience in these areas, we believe we are the best fit for this job.
We have multiple years of experience with Java and web scraping which makes us well-placed to tackle this project. Moreover, we use Spring Boot to make sure our solutions are easy to set up and implement. Our commitment to quality also comes through in stage + production checks which we conduct before payment.
If you're interested in working with us on this project, please let us know so we can get started on the discussions needed for closure!
Hello, Darko P.
It's a pleasure to connect with you.
Having thoroughly reviewed requirements of "Java scraping expert required", I have a clear understanding of your expectations.
✅ I am committed to delivering a top-quality product and ensuring 100% satisfaction.
✅ You need not spend precious time searching for another freelancer; I am here to assist you.
✅ With a proven record of exceeding customer expectations and consistently adding value to products, I bring innovative and strategic thinking to every project.
I am eagerly looking forward to discussing further details in our chat.
Best regards. John
Hello sir, I have extensive experience in web scraping using Java, with a strong background in spring boot and web crawling. I can help you achieve your project's goals, including creating a crawler that scrapes content based on URL regex, stores it in folders, and takes screenshots for offline viewing. I can also handle the implementation of a spring boot cron job for scheduled scraping.
I understand the importance of creating an offline browsable version of the scraped content, and I'm well-versed in using tools like Selenium and JSoup for web scraping tasks.
I'm open to further discussions, NDA signing, and completing a paid test task to demonstrate my capabilities. Please let me know if you have any questions or need more information.
Best regards,
Ahmiii
Hello, My name is Thanh and I am an experienced Java expert with many years of experience in web scraping, Java, and Spring. I understand that you are looking for an expert to solve your requirements in Java for an offline browsable scraped content of a section of websites. Specifically, we would like the scraper to have later offline browsable scraped content and persistance so that it can be used later without needing to be online.
With my experience in Java and web scraping, I believe I can provide the best solution for this project. My past projects have all used open source software so there are no worries about security or privacy issues. Additionally, my work is always delivered on time and with high quality so you can rest assured that you will get great results from me.
I understand that you are looking for an expert to implement a Java scraping solution. Specifically, you are looking for someone who can make later offline browsable scraped content of a section of websites. Your goal is to have later offline browsable scraped content so that the content can be used offline without any connectivity issues.
With my extensive experience in web scraping and Java, I believe I am the perfect fit for this project. My background includes multiple years of experience with web scraping as well as Java. Additionally, I have multiple years of experience with spring boot which makes it easy for me to work on your project at all stages from planning through deployment.
I would be more than happy to discuss further how I can help you achieve your goals for this project if given the chance. Please let me know if you'd like to discuss further or have any other questions about my skills or portfolio? Thank you!
Hello.
I just checked your project carefully.
I am an expert in Java with over 9 years of the experience and I have a solid understanding of Apache Maven, iBATIS, Spring boot, Hibernate, JPA, Apache Common IO, Apache POI etc.
According to my experience, Spring Boot integrates well with ORM frameworks such as Hibernate and JPA (Java Persistence API) to manage the persistence layer in terms of object-to-relational mappings, transaction management, etc.
I also made a lot of Server maintenance utilities with Java.
Your requirements are ideal match for my skills and experience.
If you hire me, you will get perfect results.
I hope work hardest for your success.
Thank you.
Hello.
I have experience in Java programming.
And web scraping and automation experience with Java selenium, etc.
I can start immediately with your fair budget.
I want to discuss the details with you over chat.
Thanks.
