
Closed
Posted
I’m building a backtesting engine and now need a robust scraping solution that feeds it three streams of information: 1. Clean, timestamp-aligned OHLCV data. 2. News, social-media, or other sentiment signals tied to the same symbols and dates. 3. Company financials, earnings releases, and key macro-economic numbers. Coverage must span US stocks and ETFs, cryptocurrencies, and major forex, commodity, and futures contracts. All material should come from freely accessible public websites or APIs you select, so part of the job is choosing reliable sources that respect their own terms of service. Here’s what I expect at hand-off: • Python (preferred) scripts or notebooks that automate the collection, parsing, and normalization of each data type. • A lightweight scheduler or CLI that lets me pull fresh data on demand or run daily batches. • Output delivered as CSV files or a simple SQLite/PostgreSQL database, with consistent symbol naming and UTC timestamps so I can drop it straight into my backtester. • Clear documentation explaining the data sources, any rate-limit handling, and how to extend the scraper when new tickers or markets are added. • A brief demo run showing sample results for one equity, one crypto pair, and one futures contract. Please use well-supported libraries (requests, BeautifulSoup, Selenium, Scrapy, pandas, etc.) and keep external dependencies minimal. Error handling, retries, and polite throttling are important—this must run unattended without breaking site rules. I’ll review the code for readability, rerun the demo, and validate that fields line up across datasets. Once everything passes those checks, the project is complete.
Project ID: 40802102
151 proposals
Remote project
Active 17 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
151 freelancers are bidding on average $19 USD/hour for this job

Hello, I am Dr. Rajesh Rolen, PhD in Computer Science & Engineering, with experience of over 20+ years in PHP, Web Scraping, Software Architecture, Python As a preferred freelancer in the top 1%, I have done 400+ projects here on freelancer.com, I have 4.9 ratings out of 5 on average, which showcases my quality of work and timely delivery. Key Highlights: - Free Hosting Support on the Cloud or any desired platform. - Free 3 months of post-delivery support to ensure that our client doesn’t face any challenges after the launch of the project. - Free Dedicated tester on projects to ensure quality delivery, so clients don’t need to act as a tester. - 10+ Years experience UI/UX team to ensure intuitive UI. Portfolio: https://www.freelancer.com/u/Microlent Please open the chat and send me a message, so we can have a more detailed discussion about the project to give you the project timeline and cost. Thank you for considering my services. I look forward to engaging in a productive conversation and understanding how I can be of assistance in bringing your project to life. Regards Rajesh Rolen
$15 USD in 40 days
9.5
9.5

Python data scraping and software architecture are central to my work, supported by more than 18 years in web and app development and experience delivering solutions for over 743 clients. This makes me well suited to build a dependable multi-market data pipeline that collects, parses, and normalizes market, sentiment, financial, and macroeconomic information from publicly accessible sources. My Python skill set aligns directly with your preferred implementation, while my data-scraping background supports robust handling of structured and semi-structured websites and APIs. I can deliver readable automation with consistent symbols, UTC timestamps, CSV or SQLite/PostgreSQL output, retries, error handling, polite throttling, and clear documentation for future ticker and market extensions. My software architecture experience also helps keep the collector organized and maintainable as your backtesting engine grows across equities, ETFs, crypto, forex, commodities, and futures. I would be pleased to deliver the scripts, scheduler or CLI, documentation, and representative demo outputs needed for you to validate the pipeline confidently.
$20 USD in 40 days
9.0
9.0

The key challenge is making these streams join reliably—not just collecting them. I’ll define a symbol registry and UTC date rules first, then normalize prices, event timestamps, and financial reporting periods into documented tables your backtester can read directly. I’ll build a Python CLI for on-demand and daily pulls, using public endpoints where permitted: market-data sources verified for each asset class, Kraken for crypto, SEC filings for US company financials, FRED for macro series, and GDELT for dated news signals. I’ll document gaps rather than invent volume for forex or imply that restricted social data is freely available. Futures contract identifiers and any continuous-series roll rules will be explicit. The hand-off will include CSV or SQLite output, incremental updates, retries, rate-limit handling, source notes, and a reproducible demo for an equity, crypto pair, and futures contract. I’ll also include checks for duplicate bars, missing dates, and mismatched symbols. Do you have a preferred bar interval and a specific futures contract for the demo?
$20 USD in 40 days
8.2
8.2

⭐⭐⭐⭐⭐ Robust Data Scraper for OHLCV, Sentiment & Financials ❇️ Hi My Friend, I hope you are doing well. I checked all of your project requirements and I can see you are looking for a robust scraping solution. You have no need to look any further as Zohaib is here to help you! My team is already doing 50+ similar projects for financial data scraping. I will build Python scripts that pull OHLCV, sentiment and financial data from public sites, align timestamps and store them in CSV/SQLite. I will also add a simple CLI scheduler and full docs, all within your budget. ➡️ Why Me? I can easily do your scraper as I have 5 years of experience in Python, web scraping, data parsing, API integration, pandas, and automation. Not only this, I have a strong grip on SQLite, PostgreSQL, and scheduling tools. ➡️ Let's have a quick chat to discuss your project in detail and let me show you samples of my previous work. Looking forward to discussing with you in chat. ➡️ Skills & Experience: ✅ Python ✅ Requests ✅ BeautifulSoup ✅ Selenium ✅ Scrapy ✅ Pandas ✅ CSV Handling ✅ SQLite ✅ PostgreSQL ✅ Scheduler CLI ✅ Error Handling ✅ Rate‑Limit Management Waiting for your response! Best Regards, Zohaib
$17 USD in 40 days
8.2
8.2

I can help build a structured Python data collection pipeline for your backtesting engine, with consistent symbols, UTC timestamps, and normalized datasets. My experience includes API integration, PHP/MySQL, backend development, and data processing. I can build Python scripts using requests, pandas, and BeautifulSoup where permitted, with retries, rate-limit handling, logging, and a CLI or scheduler. I’ll organize the pipeline into OHLCV, sentiment/news, and financial/macro data collectors. I’ll prioritize documented free APIs and public sources, check usage restrictions, and avoid scraping sites where automated collection is prohibited. Data can be stored in CSV or SQLite, with clear documentation and demo outputs for a stock, crypto pair, and futures contract. One important point: free sources may not provide complete historical coverage or reliable sentiment data across every market, so I’ll document source limitations rather than assume uniform availability. Which futures markets are essential for your first release: CME, ICE, or both? Best regards, Jafarali Maknojiya
$18 USD in 40 days
7.9
7.9

Hi, I can help you with "Multi-Market Trading Data Scraper" as per your given project description. We can discuss more in detail during a chat conversation when you are available. I've worked on many PHP projects in recent times. So I am confident on achieving your expected Goals. Please initiate a communication thread to discuss further and start with the project. ⭐ 5.0/5 from a recent client: "Excellent work. Would definitely utilize again. Stellar job!" Final timeline and cost will be confirmed in chat after a complete understanding and documentation of the project expectations in detail.
$15 USD in 7 days
7.7
7.7

Hi, I can help you build a robust scraping solution for your backtesting engine. With my 4 years of experience in Python development and web scraping, I will design scripts that collect clean, timestamp-aligned OHLCV data, sentiment signals, and company financials from reliable public sources. I will implement a lightweight scheduler or CLI for on-demand data pulls and daily batches, ensuring the output is delivered as CSV files or a simple SQLite/PostgreSQL database with consistent symbol naming and UTC timestamps. I will utilize well-supported libraries such as requests, BeautifulSoup, and Scrapy, ensuring the solution handles errors, retries, and adheres to site rules for unattended operation. My delivery will include clear documentation on data sources, rate-limiting strategies, and extensibility for new tickers or markets, along with a demo run showcasing results for an equity, a crypto pair, and a futures contract. Could you specify any particular websites or APIs you prefer for data sourcing? Also, how often do you expect to run the data collection? Looking forward to your response. Best, Hussein.
$25 USD in 5 days
6.5
6.5

Good day! With 12 years of experience, 475+ completed projects, and as a Top 1% Verified Freelancer, I’m a seasoned and practicing Python data scraping specialist. I focus on building reliable data pipelines that collect, normalize, and organize financial market data for analysis and backtesting. Here’s how I’ll help: • Collect and normalize OHLCV data for stocks, ETFs, crypto, forex, commodities, and futures. • Integrate public news, sentiment, company financials, earnings, and macroeconomic data sources. • Standardize symbols, field structures, and UTC timestamps across datasets. • Build Python scripts with retries, error handling, rate-limit management, and polite throttling. • Add a lightweight CLI or scheduler for on-demand and daily data collection. • Export results to CSV or SQLite/PostgreSQL with clear documentation. • Demonstrate the pipeline using an equity, crypto pair, and futures contract. I’ll prioritise reliable public APIs and sources whose terms permit the intended collection and use. I’ll also document source limitations, data coverage, and any licensing or historical-data restrictions so the backtesting workflow remains transparent and reproducible. Do you have a preferred database, or should I use SQLite for the initial lightweight implementation?
$20 USD in 40 days
6.3
6.3

I can build your backtesting data pipeline to collect, parse, and normalize OHLCV data, sentiment signals, and fundamentals across US equities, crypto, forex, commodities, and futures. Using Python, Pandas, and requests, I will set up reliable scrapers and API integrations that respect rate limits with built-in retries and polite throttling for unattended daily runs. The system will include a lightweight scheduler or CLI, outputting clean, UTC-timestamped data directly into CSV files or a SQLite/PostgreSQL database with consistent symbol naming. I will also provide clear documentation on sources and rate-limit handling, alongside a demo run covering an equity, a crypto pair, and a futures contract so you can immediately validate that fields line up across your datasets. Let's discuss how to structure the sources and database schema for your backtester.
$15 USD in 7 days
6.1
6.1

The real risk here isn't scraping, it's alignment. Crypto trades 24/7, futures roll monthly, and equities adjust for splits, so naive UTC merges create look-ahead bias that quietly inflates your backtest results. Quick questions - do you want continuous back-adjusted futures series or individual contracts with roll dates? And what bar granularity do you need, daily only or intraday OHLCV? Here is the architectural approach: - DATA SOURCING: Pull OHLCV from free APIs like Binance, Stooq, and FRED, with one Python adapter per source so new markets plug in cleanly. - NORMALIZATION: Pandas maps symbols to one scheme, converts to UTC, and tags financials by release time to block look-ahead
$18 USD in 30 days
6.5
6.5

Hi, I can build your multi-market scraper in Python using BeautifulSoup, Scrapy and pandas, with a scheduler and CLI for daily pulls and clean CSV or SQLite output with UTC timestamps for your backtester. I will add retries, rate-limit handling and clear documentation so you can extend it to new tickers and markets. Happy to start with a demo run for one equity, one crypto pair and one futures contract.
$16 USD in 40 days
6.3
6.3

Hello, Your backtester needs three feeds that actually line up on the same symbols and dates, not three separate dumps that you have to stitch together later. I would build this as a small Python pipeline with one shared normalizer so OHLCV, sentiment, and fundamentals all land with the same ticker map and UTC timestamps before they ever hit CSV or SQLite/Postgres. For prices I would lean on stable public APIs and exchange endpoints where the terms allow it, then fill gaps with careful HTML parsing only when needed. Sentiment would come from public news and social pages tied to those same symbols. Financials and macros would pull from free filings and economic calendars. Each source gets its own adapter, polite throttling, retries, and clear logging so overnight runs do not trip rate limits or leave you with half a day of data. You get a CLI plus a light daily scheduler, docs on every source and how to add tickers, and a demo for one equity, one crypto pair, and one futures contract so you can drop the output straight into the backtester and check the joins yourself. I can put in 40 hours each week on this and keep dependencies lean (requests, BeautifulSoup, pandas, optional Selenium only where a page truly needs it). Happy to walk through source choices before we lock the...
$20 USD in 7 days
6.0
6.0

Hi - Truong here, "ALIGN EVERY MARKET DATA STREAM TO ONE TRUSTWORTHY UTC TIMELINE" - your backtester needs more than scraped numbers; it needs consistent symbols, timestamps, and data structures across markets and sources. I’d build this in Python with separate collectors for OHLCV, sentiment/news, and financial or macroeconomic data. Each source would have its own parser and normalization layer, converting timestamps to UTC and mapping symbols into a consistent format. I’d prioritize public APIs where available, use scraping only where permitted, and include throttling, retries, validation, and logging to keep unattended runs reliable. The delivery would include a CLI or scheduler, CSV or SQLite/PostgreSQL output, source and rate-limit documentation, and a demo covering one equity, crypto pair, and futures contract. I’d also flag coverage gaps where free sources cannot reliably provide the requested history. Which historical depth and update frequency do you need, and do you have a preferred storage format? Looking forward to working with you.
$20 USD in 40 days
5.9
5.9

Python backend development and database architecture make me well suited to build a reliable multi-market data pipeline for your backtesting engine. I can create maintainable collection and normalization scripts for OHLCV, sentiment, financial, earnings, and macroeconomic data, with consistent symbols and UTC timestamps across equities, crypto, forex, commodities, and futures. My experience in REST API integration, software architecture, PostgreSQL, and database management directly supports the required CSV or database output. I can also deliver a lightweight CLI or scheduler, structured error handling, retries, throttling, and clear documentation covering sources, dependencies, rate limits, and future ticker expansion. Python is included among my core development skills, allowing the solution to remain readable and practical to operate. I am ready to deliver a documented, production-minded scraper with a demo covering an equity, crypto pair, and futures contract. Choose me to build a dependable foundation for your backtesting workflow.
$15 USD in 40 days
5.5
5.5

Building reliable automated data pipelines is a core strength of mine, combining Python, web scraping, data analysis, and software architecture to turn public web sources into structured, usable datasets. This aligns directly with your need for synchronized market data, sentiment signals, financial information, and macroeconomic figures for a backtesting engine. My web scraping experience supports collecting data across different source formats, while Python and data analysis are well suited to parsing, normalization, consistent symbol handling, and UTC timestamp alignment. I can also structure the solution as maintainable scripts with unattended execution, error handling, retries, throttling, and CSV or SQLite output. My software architecture background helps keep the code readable and extensible as additional markets, tickers, and sources are introduced. I would be glad to deliver a practical scraper package with documentation and a representative demo covering equity, crypto, and futures data. Choose me for a careful, scalable implementation focused on reliable outputs and straightforward maintenance.
$20 USD in 40 days
5.6
5.6

Hello, You need a Python scraping pipeline that collects aligned OHLCV, sentiment, and fundamental data across equities, crypto, forex, and futures into SQLite or CSV. I will build modular Python scripts using requests, BeautifulSoup, and pandas to pull and normalize market data into a SQLite database with UTC timestamps. The tool will include a CLI scheduler, rate-limiting, retry logic, and clean source documentation. Deliverables: - Python scrapers and CLI scheduler for OHLCV, sentiment, and fundamentals - SQLite database schema and CSV exporter with UTC alignment - Setup documentation and demo script covering stocks, crypto, and futures Timeline: 10 days Cost: USD 100 for milestone 1 covering scraper architecture and demo; total estimated scope is USD 100 to USD 200 depending on final website selections. Historical scraped data relies on source uptime and target site structure remaining stable. Which specific free public sources or APIs do you prefer for futures and sentiment signals? I can share sample SQLite outputs from previous Python scrapers in chat. Thanks, Prashant
$100 USD in 40 days
5.6
5.6

⚠️ If you're not happy, you don’t pay. ⚠️ Hi Client, thank you for checking my proposal and sharing the detailed project brief. I can build your robust scraping solution using Python with a clean, efficient, and scalable design. I will deliver: • Automated scripts for OHLCV data collection from reliable sources • Sentiment signal scraping from news and social media • Financial data extraction for companies and macro-economic indicators • Lightweight scheduler for on-demand and daily batch data pulls • Output in CSV or SQLite/PostgreSQL with aligned timestamps • Comprehensive documentation covering data sources and extensibility • A demo run showcasing results for equity, crypto, and futures You will also receive clear guidance on error handling and best practices for future expansions. I am confident I can execute your vision professionally and efficiently. Looking forward to discussing the timeline and next steps. Best regards, Chirag Pipal
$15 USD in 14 days
4.9
4.9

I start today by joining public price, sentiment and financials sources on symbol plus UTC date into one Python collector, output CSV and SQLite, retries and throttling included. Past job: Selenium scraper.
$20 USD in 40 days
5.0
5.0

Python, web scraping, and software architecture are central to TechWagger’s work, making this project a strong match for our team. We can build maintainable collectors for market prices, sentiment, financial data, earnings, and macroeconomic indicators while keeping symbols, schemas, and UTC timestamps consistent across asset classes. Our Python and web-scraping capabilities support the automation you need, including parsing public websites or APIs, structured CSV or SQLite/PostgreSQL output, retries, rate-limit handling, and unattended execution. Our software architecture experience also helps keep the solution extensible, so additional tickers, markets, and data sources can be incorporated without rewriting the entire system. Clear source documentation and a representative demo covering equity, crypto, and futures data will be included. TechWagger is ready to deliver a readable, practical data pipeline that can connect cleanly to your backtesting engine. Choose us to build a dependable foundation for your multi-market research workflow.
$21 USD in 40 days
5.0
5.0

Three streams feeding a backtesting engine means you need scrapers that stay in sync and handle rate limits without dropping data. I'd use Python with BeautifulSoup for the static sources and handle pagination carefully, storing everything in a clean MySQL schema your engine can query directly. Can start today, first stream running within 48 hours. These numbers are based on the post as written, we'll refine both after a quick scope talk. Want me to send a plan?
$25 USD in 7 days
4.6
4.6

Nakuru, Kenya
Member since Nov 4, 2026
$3000-5000 USD
$30-250 USD
₹12500-37500 INR
$15-25 USD / hour
$30-250 USD
$3000-5000 USD
£18-36 GBP / hour
₹1500-12500 INR
$15-25 USD / hour
₹75000-150000 INR
₹1500-12500 INR
₹600-1500 INR
$8-15 USD / hour
₹12500-37500 INR
$250-750 USD
$250-750 USD
$25-50 USD / hour
₹1500-12500 INR
$250-750 USD
$250-750 USD