Pulling bulk market data from Polygon

No REST endpoints. Just data — structured, complete, and yours

Jared Bennett

May 06, 2025

The long-short of it

REST APIs break. Slow, fragile, rate-limited. Not built for scale.
Polygon’s flat files don’t. Three daily CSVs. Full U.S. market. No queries.
MinIO makes it a one-liner. Set it up once. Run it forever. Direct, fast, reliable.

The problem

You need market data. Clean. Daily. Dependable.
Not tangled in API keys, rate limits, or connection timeouts.

At some point, scraping Yahoo Finance or using yfinance becomes more pain than progress. IP blocks. Timeout errors. Too fragile to trust. Too slow to scale.

If your data is foundational — then so is your pipeline.
You don’t need a workaround. You need infrastructure.

The fix: Flat Files + MinIO

Just three commands. Three files. Full-market coverage.

Polygon.io skips the REST API dance.

No REST endpoints. No rate limits. Just flat files — published daily to S3-compatible cloud storage.

You get: prices, splits, dividends — all timestamped and aligned.

Ideal for broad universes — S&P 500, Russell 2000, or your own basket.
No polling. No ticker loops. Just data — structured, simple, ready.

What you’ll need

A Polygon.io account with a paid plan (e.g., Stocks Starter – $29/month)
The MinIO Client (mc) — a fast, zero-friction command-line tool
Your Access Key and Secret Key from the Polygon dashboard

Setup guide (macOS)

1. Get Your API Keys

Take note of:

Access Key
Secret Key

2. Install MinIO (mc)

In Terminal, run the appropriate command:

Apple Silicon (M1/M2/M3/M4):

curl -O https://dl.min.io/client/mc/release/darwin-arm64/mc

Intel Macs:

curl -O https://dl.min.io/client/mc/release/darwin-amd64/mc

This will download a file named mc into your current folder.

Then make it executable:

chmod +x mc

And move it to your global path

sudo mv mc /usr/local/bin/

You’ll be prompted for your password. This makes mc available anywhere in your Terminal.

3. Verify the Install

mc --help

You should see a list of supported commands.

4. Link MinIO to Polygon

mc alias set s3polygon https://files.polygon.io YOUR_ACCESS_KEY YOUR_SECRET_KEY

Replace with your actual credentials (YOUR_ACCESS_KEY and YOUR_SECRET_KEY)

That’s the setup. You’re in.

You now have flat file access.

From here, you can:

Pull historical prices, splits, and dividends;
Fetch daily updates;
Script a reliable pipeline.

No endpoints. No timeouts. No guessing.

This isn’t a REST API call

It’s direct access.

You don’t request data — you take it.
One connection. Three commands. Full coverage.

Flat files shift your pipeline from fragile to industrial.

When everyone else is rate-limited or erroring out —
you’re already processing.

This is infrastructure edge.
This is process alpha.

Welcome to Point276