Web crawler aggregatelavori

Per progetto in partenza a breve, si cerca un/una DATASTAGE attività che dovranno essere svolte in DS: 1 - Acquisizione su Data Hub (popolamento base dati direzionale) 2 - Esposizione su Presentation Layer (predisposizione viste Denodo, popolamento repository cdg e creazione viste aggregate per reporting) Prj Integrazioni BD - Bracco 1) realizzazione ETL per l’integrazione in Dwh Mercato di 8 tabelle esposte da Datahub e creazione delle relative tabelle. 2) realizzazione degli ETL per l’esposizione delle 8 tabelle nuove e di altre 11 tabelle del dwh primario e di 26 datamart tramite creazione di viste 3) disegno modello dati (schema excel) 4) predisposizione schedulazione per esposizione viste - ETL di Integrazione di una tabella da Datahub con chiave abi/nag ...

Buongiorno, vorrei creare sul mio sito una pagina/motore di ricerca che cerchi tramite campo di ricerca, un prodotto su 4/5 siti specifici. Una versione semplificata di trovaprezzi. Dato il nome dell'oggetto da cercare mi fa vedere un grafico dell'andamento dei prezzi nel tempo, e il prezzo del prodotto nei 4 siti indicati. Tutto cio aiuta l'utente finale a capire se è un buon momento per acquistare. Non mi servono affiliazioni etc... una semplice ricerca con estrapolazione dei dati. Cerco una stampante marca e modello xyz, mi fa vedere il grafico del costo del prodotto nel tempo, e il prezzo in questo momento sui 4 siti indicati. Sul sito A costa 10 sul sito B costa 8 e cosi via.

Ho bisogno di realizzare un crawler che acceda ad un sito mediante login (credenziali salvate, no captcha) e acceda ad una determinata sezione del sito e recuperi i dati di alcuni clienti (cognome, nome, telefono, email, ed altre 3/4 informazioni) e li scriva in un db. per la scrittura nel db forniamo noi una pagina web che questo programma chiamerà passando i parametri (cognome, nome etc, attraverso post) e la pagina provvedera ad inserirli fisicamente nel db e restituirà (de definire in che modo insieme a chi farà il programma) l'esito dell'operazione.

Buongiorno, Sono un rivenditore di prodotti elettronici ed attualmente vendo prodotti da diversi csv tedeschi. Ho un grosso progetto da sviluppare, ovvero prelevare con qualche sistema Crawler o simile un csv dal sito che ha degli ottimi prezzi ma non vende all'estero come altri, in pratica non è un b2b. Dispone solo di codici produttori che però grazie ai miei listini csv di altri fornitori possono essere in parte convertiti in EAN e quindi essere poi rivenduto su Marketplace come Amazone ecc. Quello che mi serve è creare il csv ed aggiornarlo tutti i giorni logicamente. Attendo Riscontro Cordiali Saluti Massimo Burlando

craiglist phone number scrapper

Un web crawler che partendo da uno o piu URI scarica le relative pagine e fa lo stesso per i link contenuti nelle pagine scaricate (maggiori info nel package)

Salve. Cerco qualcuno che programmi un SEMPLICE (MOLTO SEMPLICE) Bot web crawler per un motore di ricerca (il motore di ricerca sarà per un sito interno, quindi non deve ricercare tutte le pagine del web)

dato come input un folder contenente una serie di data sets (.csv files) fare il loop sulla cartella per eseguire su ognuno di essi una serie di operazioni: in particolare creare delle sliding windows basate sul valore della prima colonna con dimensione variabile e restituire per ogni window il valore medio delle altre colonne.

L'app Android serve a monitorare la movimentazione di un gruppo di colli da una sede A ad una sede B. Il collo ha un'etichetta che lo identifica, tramite il codice identificativo del collo l'applicazion...un gruppo di colli da una sede A ad una sede B. Il collo ha un'etichetta che lo identifica, tramite il codice identificativo del collo l'applicazione aggiorna su un database lo stato della movimentazione di tutti i colli. Il progetto prevede anche la creazione di uno script (ad esempio php, ma e' a discrezione dello sviluppatore) che gestisca le utenze ed il DB (ad esempio Mysql) dove vengono aggregate le informazioni inviate dai dispositivi Android sul campo. Sono disponibili i diagrammi di flusso di dettaglio di tutte le funzioni, in modo da agevolar...

I am seeking an expert capable of creating a highly detailed and intricate graphic line art rendition of a lattice boom crawler crane with a luffing jib (two independently controlled booms). This rendition must strictly capture the exterior features of the equipment ( there's a lot). Your focus should also be on all the components that make up the crane. I am open to starting with a more basic line art (outline) and seeing what we need to add. This would be the first design for a construction apparel line You need to have a meticulous eye for detail and at least have a portfolio that showcases any automotive art.

I am looking to create a sports news aggregation website. The website will be primarily focused on sports news and will be used as the basis for a radio broadcast. Key Features: - The primary function of this site is to aggregate sports news from a range of different sources. These sources may include major sports news websites, blogs, newspapers and more. This content will form the basis of our daily radio show. - The site should also have a feature where the latest sports news can be read out loud. This is a key feature as the site will be used for our radio broadcasts. - The website should include a section that is dedicated to sports video highlights. This section will be a key engagement tool for our audience. Ideal Skills and Experience: - Experience in creating news aggrega...

I need a skilled professional should know very very good knowledge in K6 Broswer performance automation testing and reporting. should not use API for performance. If you know Browser automation with Html and Graphana report is enough. Just need to automate below test cases with html report and Graphana report for 10 vus in local and 100 in Grap...1000$ is dispayed Verify how much time it taking to complete the payment. * create simple page object model framework * should run 10 vus in local * Add Testng Report if possible * how many test cases executed how many passed or failhow many test cases executed how many passed or fail * need to write this code in javascript using k6. * need matrix how much time is taking for each page load (not Aggregate matrix which k6 tool is giving b...

...of creating a customized web crawler for eBay and AliExpress. The primary function for this crawler will be to accept a list of keywords of any length and return specific data about those products. Here's what I need in detail: • Find top hot selling products at any given time • Crawl eBay and AliExpress for specific product data based on entered keywords. • The data retrieved should include product names, quantities sold and locations of sellers, images, • The crawler should be capable of providing this information for a variable timescale, ranging from 1 to 30 days in the past. • The results should be sortable. Php or python preferably The task would best suit a freelancer with previous experience designing similar cr...

I am in need of an application which can streamline my client management process. Key Features: - **Messaging Aggregation:** The app should be able to integrate WhatsApp and possibly other platforms to aggregate messages from my clients. - **Client Management:** The core function of this app is to offer a comprehensive client management solution. - **Client Contact Information Storage:** It must have a feature to store client contact information in a structured manner. - **Client Interaction History:** The app should record and display the history of client interactions. - **Appointment Scheduling:** A feature should be available to schedule appointments with clients. Ideal Skills and Experience: - Proficient in app development with experience in creating integrated solutions. -...

I am in need of an application which can streamline my client management process. Key Features: - **Messaging Aggregation:** The app should be able to integrate WhatsApp and possibly other platforms to aggregate messages from my clients. - **Client Management:** The core function of this app is to offer a comprehensive client management solution. - **Client Contact Information Storage:** It must have a feature to store client contact information in a structured manner. - **Client Interaction History:** The app should record and display the history of client interactions. - **Appointment Scheduling:** A feature should be available to schedule appointments with clients. Ideal Skills and Experience: - Proficient in app development with experience in creating integrated solutions. -...

I'm looking for a skilled professional who can crop the complete content of a website and save it in a zip file to be deployed on a new server. The website is a small Wordpress with about 41 pages (as per sitemap). Content must not be saved as Wordpress, but static html to be deployed on another server. Key Requirements: - Crop all website content including subpages. - Save the cropped content in a compressed zip file. - The purpose of this cropping is to save the content for future use. - all files must be included, esprecially but not limited to images, js, css, ... - inner paths must be relative paths. If you're confident in your ability to deliver this and can ensure all the content is correctly saved, please bid accordingly. IMPORTANT READ THIS: Auto-generated bids wil...

...form and will need to be manipulated to gather the relevant data, and then display this data on graphs. For the graphs, the X axis will be time, and should be adjustable (i.e. I should be able to narrow or expand the timeframe I’m looking at). Since the logging is provided every 30 seconds, and I will likely be looking at the data on a daily, weekly, or even 2-3 month basis, the ability to aggregate / reduce the number of points on the graph will be important to ensure the graph is not slowed down / laggy. The Y axis should be one of 5 variables (and I should be able to look at 1,2,3,4, or all 5 variables at once). Each of these 5 variables should be plotted on their own Y axes, so they all fit on my screen. As I hover across the graph, I should be able to see the precis...

I need a skilled professional should know very very good knowledge in K6 Broswer performance automation testing and reporting. should not use API for performance. If you know Browser automation with Html and Graphana report is enough. Just need to automate below test cases with html report and Graphana report for 10 vus in local and 100 in Grap...1000$ is dispayed Verify how much time it taking to complete the payment. * create simple page object model framework * should run 10 vus in local * Add Testng Report if possible * how many test cases executed how many passed or failhow many test cases executed how many passed or fail * need to write this code in javascript using k6. * need matrix how much time is taking for each page load (not Aggregate matrix which k6 tool is giving b...

I am looking for a skilled developer to create a web-based fintech application. The main aim of this application is to aggregate bank accounts and provide a unified interface for banking. This project poses several key requirements: - Account Aggregation: The app should be able to aggregate various bank accounts. - Unified Interface: The app should provide a single, user-friendly interface for all banking activities. - Transaction History: The app should offer a clear and comprehensive transaction history feature. Moreover, security is a top priority for this project. I need the developer to ensure that the app is equipped with: - Two-factor authentication - Data encryption - Secure API integrations - Azure Integration - POWER BI Integration - Payment Integration - Y...

I'm looking for an expert to design a web crawler to extract email addresses from specific company websites. The ideal candidate should be experienced both in web crawling and email data extraction. Key Tasks: - Crawl targeted company websites for email data extraction - Ensure the accuracy and integrity of all extracted emails Skills and Experience: - Experience with web crawling and data extraction technologies - Proven track record in email data extraction - Excellent analytical and problem-solving abilities - Familiarity with common web interrogatories for more efficient data capture is an asset - Strong attention to detail in order to capture complete and valid email addresses The successful candidate will deliver a list of valid emails ex...

As the project lead, I'm seeking a highly skilled developer to construct a Scrapy web crawler for multiple sites. Although I omitted website specifics for now, rest assured we will provide the necessary details post-engagement. The ideal candidate is capable of: - Building an optimized web crawler with Scrapy - Extracting text data proficiently from various web pages, accurately and efficiently For the application, please provide: - Detailed project proposals outlining your approach Experience in web crawling design and Scrapy framework are a must for this project. Your technical capacity to generate the explained particulars is critical for success here. This is a golden opportunity to exhibit your proficiencies and contribute to an ex...

1 problem working to adjust an aggregate planning model changes to aggregate plan involve adjusting subcontracting cost and demand forecast.

Have operations based math problems that I need done for a work project. 3 problems that involve aggregate planning, inventory planning, subcontracting, production levels, economic order quantities ect. Strong excel knowledge required where you can run solver.

Task list 1) We hade a PsgSQL database with ticker data. We need to create another streaming data table with 1 min aggregate. Data table should update constantly, ideally each time ticker dateable is added with data 2) We need to write ports to the SQL data table to allow LAN connection to SQL from local network (other virtual machines connected by VM Oracle). Including Python connection 3) We need to write ports to the SQL data table to allow internet connection 4) We need instrument to add historical 1 min data to the database created on stage 1 5) We need Python script that will connect to 1 min data base and get top 5 instilments by price change from all instruments present in 1 min database Conditions: We do have a budget limit which is set to 150 USD, after reaching this lim...

I'm seeking someone skilled in MongoDB and front-end React development to work full time for a minimum of 2 months with potential for a permanent position within the company. We are a stock m...imaginative people to help bring our data to life focusing on the user experience. I need a developer who can handle the front-end development of my React application by implementing: - User authentication - Integrating API calls - Knowledge of web sockets and populating live data to the UI - Knowledge of Zoho API integration - Knowledge of the stock market and charting application integrations such as TradingView - Strong Linux skills (Ubuntu) - Merging React with a backend MongoDB to feed live data to the website. - Building MongoDB aggregate queries Knowledge of stock mar...

...data structures to merge JSON responses from various sources. This project aims to aggregate data originating from diverse broadsides into a single, simplified data response. the JSON response is coming from one API but we are not sure what will be the response and we need to make multiple calls to this API. for example we make three API call the response will be same JSON and we need to merge those three responses we need to make this generic since we are not sure what to get with the API response Key tasks, skills and experiences required include: - Merging JSON responses with a clear understanding of Java and simple data structures primarily. - Effectively use methods to retrieve JSON responses. - Aggregate the derived data to create a unified, simplified response. ...

... Allow for client selection and viewing individual client details. Present total and per-client-level energy consumption (kWh and Gas). Reveal margins for selected clients. Show monthly revenue and upcoming payment statuses for each advisor. Provide data on paid amounts in the last month and total payments received. Organizational Dashboard: Overview of total number of advisors and clients. Aggregate and advisor-level energy usage details. Financial reporting on revenue and pending payments for the organization and per advisor. Update on payment statuses (Pending, Received, etc.). Record of amounts paid at both organizational and advisor levels. Data Input and Management: Facilitate entry of advisor information and client ownership with a dropdown selection feature. Enable inpu...

...building a solution architecture to serve the requirement as below. I need a document that builds the solution architecture based on the below requirements a solution architecture and design to satisfy client needs. different design options on how to achieve a generic pipeline with pros and cons and recommendations. reusable utilities to validate, transform, enrich and aggregate data as it moves from source to consumption. a UI/UX prototype for configuring the pipeline and reporting errors (data and process) by different dimensions (data source, time, error category etc.). a project plan with list of activities to be performed for implementation of a POC within 6 weeks with 4-5 resources. you will need to have this covered in the presentation based

...automated content creation within Confluence's platform. ### Project Objectives: 1. **Data Extraction**: Write a script in JavaScript or Python to fetch content in JSON format from multiple Confluence pages. 2. **Data Parsing**: Parse tables from the fetched pages, extracting key-value pairs where the row name serves as the key, and the cell content as the value. 3. **Data Aggregation**: Aggregate all key-value pairs into a map, and then store these maps in an array to handle data from multiple pages. 4. **Data Integration**: Iterate over the aggregated data to create a single, consolidated table. Use this table to create a new Confluence page via a JSON call to the Confluence API. 5. **Testing**: Ensure the solution is fully tested on Confluence Cloud, no...

>Registrati o accedi per visualizzare i dettagli.

I require an IAM role with specific permissions to interact with s3, glue, and Athena on AWS. This role will allow for efficient management of the AWS stack, specifically enabling the creation of a crawler to populate a glue catalog database and tables, and also allowing interaction with an Athena workgroup. What I need: - Read-only access: This permission should allow the IAM role to view S3 bucket contents, view Glue catalog information, and view Athena workgroup details. - Write access: This level of permission should enable the IAM role to create or update tables in the Glue catalog and modify Athena workgroup settings. Ideal Experience and Skills: - Solid understanding of IAM roles and permissions on AWS - Experience with AWS s3, Glue, and Athena - Capability to work on AWS I...

I need an experienced and proficient website crawler who can help me extract Text content from Database Tables on my website. Key requirements: • Acing in finding, fetching, and structuring data. • Proficient in delivering the retrieved data in Plain text format. • Emphasis on fetching Text content from the Database Table, thus understanding of database structures is a must. Ideal skills include web scraping, data mining, and database understanding. It would be best if you had an eye for detail as accuracy is key in this project. With your expertise, you should be able to navigate through complex website layers to get to the desired content.

...recent traded prices to educate the user on how much a product is currently worth. Now, with the help of data scraping & AI, we can first grab price data on other platforms such as Amazon, Gumtree and eBay so we have sufficient data entries to input for the AI to work. With AI, we can take this one step further. First we analyse all the pricing data on Flipspot and other platforms. Then we aggregate the data and feed it into our AI Algorithm to create real-time pricing tools. Some of these features include: • Pricing trends: What products are currently trading at a discount? What products are currently trading at a premium? • Pricing Recommendations: What price is considered a bargain when buying? What price is considered a great price when selling? • Market...

We are in need of a knowledgeable and skillful programmer who can edit on GoLang code. The main task involves Ethereum network crawling. We already have a running code to crawling the Ethereum network. However, we still need to do some changes on GoLang code (designing crawler), such as partition the ID_space into sections, and crawl using multiple crawlers in Ethereum not only single crawler as what we have. In addition, there might be more few changes on the code. The ideal candidate will: - Understand DHT-based network and Kademlia - Knows a little about Ethereum (blockchain) - Have experience in GoLang - Possess a deep understanding of DHT network crawling (p2p) - Have previous experience with crawling networks Your responsibility will be building on top of running code...

We are in need of a video editor that can aggregate clips from various sources and integrate them with the voiceovers that we provide. The videos should somewhat look like these We need video every day, we will provide voice overs and you need to make a 8-10minute video. And for every such video You will be given 10-12usd. So monthly you will be provided with 375-400 usd.

1) We hade a PsgSQL database with ticker data. We need to create another streaming data table with 1 min aggregate. Data table should update constantly, ideally each time ticker dateable is added with data 2) We need to write ports to the SQL data table to allow LAN connection to SQL from local network (other virtual machines connected by VM Oracle). Including Python connection 3) We need to write ports to the SQL data table to allow internet connection 4) We need instrument to add historical 1 min data to the database created on stage 1 5) We need Python script that will connect to 1 min data base and get top 5 instilments by price change from all instruments present in 1 min database Bot check: write price for each part from 1-5

...ranging from agriculture, outdoor event management, hospitality, travel and tourism, and healthcare. While GTM has no expertise in meteorology, it proposes to aggregate weather data from multiple providers and use analytics to correlate it with meaningful conclusions for businesses. Example use cases could be generating forecasts based on local weather conditions around: 1. Consumption of hot versus cold beverages depending on cold, sunny or rainy weather 2. Number of visitors to a tourist site or an open-air entertainment event 3. Likelihood of seasonal illnesses such as flu in particular locations The main system will comprise a web portal and a set of “apps” available on the popular mobile operating systems. Apart from this, clients can ask for specific serv...

...Romanian need to be translated into English. Additionally, a reclassification of these statements may be necessary to align with international financial reporting standards. 2. **Creation of an Aggregate Financial Statement**: Compile the financial data from both the parent and subsidiary companies into a single, comprehensive financial statement. 3. **Valuation**: Conduct a thorough valuation of the group based on the provided financial statements and other relevant financial data. 4. **Data Pack Creation**: Develop a final data pack that includes the reclassified financial statements, aggregate financial statement, valuation, and other relevant financial information. This data pack will be used as a key document in the sale process of the group. **Requirement...

Hello, My name is Kaylen Kerr and I am an environmentalist working for Terus Construction. We specialize in asphalt, concrete, and aggregate products and we are in need of a professional biologist to do a migratory bird nesting survey at 3 of our gravel mines located in the Kootenay region of B.C between Slocan and Trail. We will not be doing any intensive logging at these sites and will mainly be using the sites for aggregate crushing, however, we may need to fall a few trees in order to expand parts of the pits. Ideally, we would like to have these surveys completed by early April. Thank you,

I am looking for an experienced data analyst who is well-versed in PySpark to clean up a medium-sized dataset in a CSV file format. The file contains between 10k-100k rows, and your primary role will be to: - Remove duplicate data entries - Deduplicate the dataset - Handle missing values - Aggregate the resultant data Your proficiency in using PySpark to automate these processes efficiently will be critical to the success of this project. Therefore, prior experience in handling and cleaning similar large datasets would be beneficial. Please note, this project requires precision, meticulousness, and a good understanding of data aggregation principles.

I'm seeking a skilled professional with expertise in constructing concrete driveways and slabs. The project involves: - Laying an approximately 75 ft driveway and a 30’ x 75' slab for a barndominium. - Applying an exposed aggregate finish for a robust and appealing aesthetic. - Working with clay soil, requiring knowledge of appropriate construction techniques for this soil type. Experienced contractors with a history of quality exposed aggregate work and a working knowledge of clay soil conditions will be ideal for this job.

...GAAP and Romanian GAAP. A consolidated financial statement sheet with extensive account details according to Italian accounting principles. Financial Statement Reconciliation: Adjust the Romanian company's financial statements to align with Italian accounting standards. Highlight differences and reconcile them in a dedicated column within the Excel workbook. Consolidated Financial Reporting: Aggregate the adjusted financial data into a final consolidated financial statement that adheres to ITA GAAP, complete with detailed account breakdowns. Documentation: Provide clear notes and documentation within the Excel workbook explaining the methodologies used, key accounting differences considered, and any currency conversions performed. Skills and Expertise Required: Proficienc...

Google Jobs Scraper project requiring skills in .NET, Python, Selenium and SQL. The task is to create a crawler to extract job listings based on predefined queries, manage the data in a structured manner, and provide an interface for controlling and monitoring the scraping process. The solution requires adept handling of rate limits and the use of rotating proxies for efficient data scraping. Data To Be Scraped: - Scrape all specified job listing data points. - Store and manage the scraped data in a SQL database for effective access and organization.

I'm currently facing issues with my web crawler code. Specifically, it's not managing to scrape all the required data and it's throwing errors inconsistently. This code has been built to scrape information from certain e-commerce platforms. The ideal freelancer for this task should show: - Proficiency in Node.js - Extensive experience with Puppeteer and web crawling, especially focusing on e-commerce platforms - Exceptional debugging skills Your challenge will be to inspect the code, identify the underlying issues and then fix them to ensure a comprehensive and seamless data scraping process. I look forward to collaborating with someone who can effectively resolve the issues and optimize the performance of this web crawler.

your website is causing the load spikes in question. You can consider reviewing this on your side additionally. We have also noticed that there is a lot of bot/crawler access to the website. Access requests originating from bots are registered every 2 to 5 seconds.

Hi, Today i have created a Scrapy crawler to crawl the website , but i'm not able to crawl the website because when i run the crawler, any request i do to the website, i got the 403 error to me. I just neeed you fix my code to do a request at someway the website return the code 200 instead of 403. I'm sending my code attached. Send me a print screen in the chat showing you received code 200 using the same url, then i will hire you to send me your code.

Looking for a seasoned AWS expert to assist in optimizing EC2 instances, S3 storage, and Lambda functions for improved performance. The ideal candidate will have: Mainly working on Aws glue jobs , S3,lambda, Crawler, step functions, dynamo DB ,Clouldwatchs, Appflow, parameter store , API Gateways, ETL Jobs,splunk logs CloudFormation - important big bucket,Spunk, Jenkins pipelines* CI/CD * , YAML for creating templates and manifest files * Sql mandatory for date base * ⁠using Postman for API calls * ⁠Python and spark Musk know

...developer to create a specialized web crawler focused on extracting vital data from e-commerce websites, primarily targeting ''. Key Requirements: The web crawler must adeptly parse text-based content and efficiently navigate the site's structure to extract pertinent information and links. Extensive expertise in Web crawling, Web scraping, and Data mining is essential for this project's success. Prior successful engagements in web scraping projects centered on e-commerce platforms would be greatly advantageous. Proficiency in programming Python is required. Familiarity with web scraping frameworks and libraries like BeautifulSoup, Scrapy, or Selenium is highly desirable. While the initial focus is on ...

I am seeking a professional who can provide a thorough and comprehensive summary table detailing the highlights from academic journal articles that focus on utilizing recycled aggregate, wash water, and supplementary cementitious material in concrete. You are expected to: - Focus specifically on the usage of recycled aggregate, wash water, and supplementary cementitious materials. - Organize the table by highlight category, not by article title or publication date. - Source information strictly from academic journals. The ideal candidate for this project should have: - A strong background in engineering or building materials. - Excellent research and data compilation skills. - An understanding of academic sources and how to extract key information. - Experience creating d...

I'm seeking a seasoned Full Stack Developer with expertise in Financial Application Development here are some informative requirements for developing an application: Real-time Market Data: The application should provide real-time data on stock prices, indices, commodities, currencies, and other financial instruments. News Aggregation: Implement a feature to aggregate news articles related to financial markets from various reliable sources. Customizable Watchlists: Users should be able to create and customize watchlists to track their favorite stocks and securities. Interactive Charts: Incorporate interactive charts with various technical indicators to help users analyze market trends and make informed investment decisions. Portfolio Management: Include features for users to tr...