Build data pipelines and analysis for a research paper
A researcher used Claude Code as the main agent to build data pipelines and do the data analysis for a paper, generating over 98% of the code.
Done withClaude Code
- What they did
- Kevin Yang ran Claude Code in the terminal beside Cursor and Jupyter. He had it write a plan in Markdown, then a more detailed plan for each component (cleaning, two research questions, outputs), then one script per step. He gave it parquet-tools through CLAUDE.md so it could inspect data and check its own output. He reviewed each script and used Git to roll back.
- How it went
- He estimates Claude Code wrote over 98% of the paper's code over a few weeks. It still made mistakes, so he reviewed every step. He wrote publication figures himself because describing the tweaks was harder.
- Worth knowing
- He used the Pro plan at about $17 a month on yearly billing, with Sonnet 4. When he hit rate limits he switched to his own API key.
Try it yourself with Claude Code
Help me build the data pipeline and analysis for my research project on [topic]. The raw data is in [folder or file], and I want to [cleaning, merging, and statistical tests or models]. Write clean, documented scripts, run them, and save the outputs and figures to [output folder] with a README explaining each step so I can check the results. Ask me before deleting or overwriting any existing files.
Source: yang3kc.substack.com · Undated
Five of these in your inbox every morning
The best things people got an AI agent to do, each with the prompt to try it.
More like this
Turn tickets into merged pull requests
Pulls tasks from GitHub Issues or Linear, then runs Claude Code or Codex in isolated Kubernetes pods to carry each one through to a PR.
Manage an inbox and build courses and budgets with Claude Code
A non-technical user filters their inbox to emails needing replies, turned 15 homeschool PDFs into interactive narrated courses, and built a budget dashboard.
Build a 2D platformer game without coding
A content creator with no coding experience used Claude Code to build a black-and-white wave-combat platformer in HTML canvas and played it live.
- Custom agent
Run persistent financial research with an agent
LangAlpha is a Claude Code-style agent for investing research that keeps persistent workspaces and memory so analysis such as models and comps can be updated across sessions.
Automate a weekly arXiv paper triage
A weekly Codex automation searches arXiv for keywords, logs new papers to a CSV, sorts them into keep/skim/drop buckets and can check claims against Zotero.
Catalog and summarize papers dropped into Drive
An n8n workflow extracts metadata from new PDFs, finds related papers on Semantic Scholar and PubMed, builds an APA citation, logs to Sheets and creates a Notion summary.