RFC: Using Promptless to help 10x docs contributors

[sarahxsanders]

Using Promptless to help 10x docs contributors

Deciders: @edwinyjlim @gewenyu99 @daniloc @sarahxsanders
Decision: We are going to proceed with a pilot with Inkeep's Content Writer, see more information here
Deadline: January 16, 2026

Using Promptless to help 10x docs contributors

What Promptless does

Promptless automates the creation of docs PRs based on code changes. It doesn't write or review code (like Greptile), it writes docs about code.

It works by detecting code changes, PRs, or team discussions that require documentation updates (sourced from GitHub, Slack, etc.), then opens a docs PR with drafted updates. It eliminates manual overhead of maintaining docs and ensures customers have accurate, up-to-date docs.

Capabilities

Promptless has a lot of features. I listed ones that may be useful for us:

Feature	What it does	Why this matters for us
Voice match	Custom fine-tuned models matching your docs style guide	Maintains our unique PostHog voice at scale
Auto-drafting	AI generates first drafts from code changes	Engineers review instead of write, reducing barrier to entry and scale burden
Citation transparency	Show exact sources (files, commits, convos) for each draft	Can verify sources when reviewing the AI generated content
Custom instructions	Custom generation rules	Can enforce PostHog specific patterns (templates, style rules, etc.)
Changelog publishing	Auto-creates changelog entries when features ship	Changelog takes a lot of time already. We could merge our changelog automations in Promptless for a unified management solution
Context assembly	Pulls info from GitHub for context (for example, issues)	No more hunting through tickets/threads/etc. for context
Promptless Capture	AI agents navigates product, captures screenshots, detects outdated ones, regenerates screenshots	We can have the best of both worlds: screenshots AND accurate ones at that
Screenshot editor	Built-in editor to crop, annotate, highlight, add text/shapes/arrows	Keeps screenshot workflow in one place
Slack image support	Add screenshots from Slack messages to docs	Could potentially use screenshots from changelog Slack messages
GitHub PR/commit triggers	Auto-detect when code changes need doc updates	Catches docs gaps before they become debt, keeping us proactive
Trigger refinements	Filter which repos or directories trigger	Control noise and trigger only on relevant changes
Slack Listen	Monitors channels, auto-suggests docs from conversations	Captures domain knowledge that lives in Slack
Automate CI fixes	Auto-detects and fixes failed CI checks, linter failures, Vale rules	Reduces manual cleanup, which is huge for contributors (and often why contributors don’t follow through with PRs)

Why now?

One of our Q1 goals is to 10x docs contributors. As PostHog scales, product docs need to be self-serve. To complete this goal, there's a lot of moving parts. We need to create processes that facilitate quality docs contributions and help us maintain the accuracy of published docs.

Promptless can help us reduce barrier to entry for docs contributors, increase contributor rate and quality, and automate screenshot maintenance.

Features

Promptless has features that can solve some of our biggest docs problems. I’d estimate it can get us 40-50% of the way for 10x-ing docs contributors:

Challenge	How Promptless helps
6+ products launching in 2026 (or more)	Auto-detects when new features need docs, drafts content proactively
200+ engineers shipping code at scale	Reduces engineering burden and barrier to entry, they can review AI drafts instead of writing from scratch
Screenshot maintenance at scale	Automate screenshot capture/updates as UI changes
Docs debt	Catches gaps before they accumulate, triggers on relevant PRs
Inconsistent voice and style	Voice match maintains PostHog’s docs style guide consistently

What we still own (and should)

Promptless empowers us to move fast and maintain the guardrails. We are free to focus on strategic decisions, which are decisions we should always own:

• Contributor guidelines and processes
• Templates for different doc types
• Analytics on docs health
• Doc structure, navigation
• Creating ergonomics for the docs site (UX)
• Markdown service
• Implementing linting and Vale

What would the new workflow look like?

Before going over the new workflow, I'll highlight the issues with the current workflow:

1. Engineer opens a PR for a new feature or product
2. It is the responsibility of that engineer to write the first draft of docs, and to ask the docs team for a review. Many times, engineers either don’t write docs, or don’t get a review from us - this means we either have no docs, or docs that aren’t properly reviewed
3. Engineer merges PR, either with or without a docs PR

With Promptless, the new workflow could be:

1. Engineer creates a PR for a new feature or product
2. Promptless auto-detects and drafts a doc update in the posthog.com repo, tagging the docs team for review
3. Engineer reviews the draft for technical accuracy (instead of writing it) and approves
4. Docs team does a final review and merges

How much is this going to cost?

From what I could find, Promptless charges by documentation page count, not headcount. This works as companies scale, but we have a lot of docs.

But, there could be ROI here. I am just going to roughly estimate across docs/eng teams:

• Screenshot maintenance: 5-10 hours/week (if we actually kept up with them!)
• Docs drafting: 15-20 hours/week
• Context gathering: 3-5 hours/week
• Total: ~25-35 hours/week saved

We need to know the actual price for our current number of docs pages: 882 MDX files. We might be on the higher end of pricing tiers.

To figure out if it’s worth the $$$$, I am going to need info to fill out this formula:

At a loaded cost of $Y/hour for docs team time, we’re saving $Z/month in labor. If Promptless costs less than $Z/month, it’s a ROI-positive on time savings alone.

My honest assessment

Promptless does some things well:

• Screenshot validation is needed, and quickly. This is a debt area that keeps me up at night.
• Docs drafting could help us scale better. The AI does the first pass, so Engineers don’t have to, and they can move quickly.
• Context assembly from GitHub/Slack/Google is really useful. Saves time hunting for context.

My biggest concerns are:

• Will this reduce engineer contribution or just shift the work? The best case is that Engineers review AI drafts, catch errors, and approve them. We get drafts that are 80% of the way. The worst case is Engineers rubber-stamp AI drafts without reading them and we inherit a massive backlog of unchecked docs to review.
• Our docs site is uniquely complex. Our docs site lives in a giant website repo, has tons of custom components, unique styling, and specific patterns. Promptless seems to be aimed at people using rather simple platforms, like Fern or Mintlify. Promptless was built for the majority of docs sites, and we are the minority. The risk is that we push Promptless to its limits and don’t get the full value of the platform.
• Is this a short term solution or a long-term platform? Promptless is definitely an accelerant. It multiplies our output but it doesn’t provide the foundation, all that work still comes from us (and it should!!!) It’s missing features we’ll eventually need if we want a pulse on the comprehensive health of our docs.
• Its docs testing features are lacking. It can update screenshots, but how does it validate code snippet syntax and usability? How does it test CLI commands? If we want a holistic overview of our docs health, we need to build these tools (or they need to build them).

My take

For now, it’s a promising tool. We need speed in 2026, and it supports the scale we are facing so we can focus on foundational projects. Once we have nailed these foundations in 2026, we can discuss what bandwidth we have to build more features in house.

Next steps

We have a demo scheduled for Friday January 9th. I have a list of questions I need to ask!

• How does Promptless Capture work for authenticated applications?
• What's the review workflow for captured/regenerated screenshots?
• How extensive are the customization capabilities? For example, if we have a library of templates, style guide, etc. can Promptless use this to generate all its drafts?
• How is voice match trained?
• How does this work with custom docs platforms?
• What’s on the roadmap for more docs testing/docs maintenance features? Specifically for code snippet validation (syntax, functionality, etc.) and SDK/API references?
• What's included in a custom plan? What's the actual cost for 842 pages?
• Where is our code and documentation processed and stored? What's the retention policy? Is data used to train your AI models?
• What control over data residency do we have?
• What's your SOC 2/compliance status?
• How does your platform compare to competitors such as Dosu?

Then, I suggest we sign up for their free trial and run a pilot with an Engineering team. Their free trial gives us access to Promptless for 200 pages. We should test:

• Customizing Promptless with our style guide and docs site customizations
• Implement Promptless for an Engineering team, and have them give use feedback on whether Promptless's first drafts are technically sound, and if they like the new workflow
• Test screenshot capture and push it to its limits

Alternate vendors and solutions

Promptless is currently the most faeture-rich options for our workflow and needs. However, it still doesn't fulfill the vision of a full docs health monitoring ecosystem. So, I've broekn down some alternatives and DIY approaches we could take!

Direct competitors to Promptless

There are a few platforms out there that are similar:

Platform	What it does	Strengths	Concerns for us	Pricing
Dosu	Detects merged PRs, finds related docs, auto-updates, posts summary on PR	50k+ GitHub installs, some pretty notable customers (Prisma, Strapi, LangChain), side note: I do know the CEO/could get us a solid demo	Security/SOC 2 status unconfirmed, less features than Promptless	Open source option, self service additional pricing, maybe lower costs
Swimm	Code-coupled docs with patented auto-sync algorithm	Best security posture of all of them: SOC 2 Type II + ISO 27001, self-hosting available, bring your own LLM	Focused more for internal engineering docs, recently pivoted to legacy code modernization, so I don't know how much goes into their docs features	Pricing based on number of lines of code (feel like this would add up crazy fast)
Qodo	/add_docs command auto-generates docs on PRs	Mainly a code review tool, not a docs tool (think: Greptile)	Docs is a secondary feature	Pricing based per user w/ credit limits

The competitive landscape here is small. Dosu is the closest to Promptless in functionality, but lacks some features like screenshot maintenance. It could be worth a demo if pricing is more favorable though, and gives us a foundation to at least meet the demand of engineering updates!

Building in-house

We have strict security requirements and a custom docs site built on Gatsby. DIY-ing parts of this is inevitable, because none of these platforms provide full functionality for what we want. Here's what building might look like:

Feature	What it does	What we need	Effort
PR-triggered docs drafting	Detects code PRs that need docs, drafts updates, creates docs PR	GHA, Claude API, Vector database, Prompt templates	2 weeks
Screenshot automation	Compare screenshots in docs to current app UI, alerts when stale, retakes and queues for review	Playwright, Visual regression tool, Screenshot manifest, GHA, review workflow	2-3 weeks for basic version, 3-4 weeks for something more mature
Slack integration	Monitors Slack channels for questions/discussions that should become docs, drafts content, notifies team	Slack app, Claude API, trigger logic, notification workflow	1 week or less
Zendesk integration	Detects patterns in support tickets that should become docs/FAQs, drafts content	Zendesk webhook, Claude API, pattern detection	We already have some of this built, so 1 week to hook it up to GitHub

The total effort here would be ~6-8 weeks, plus ongoing maintenance for prompt tuning, screenshot manifest updates, and edge cases. It's very possible we could build the capabilities of Promptless or Dosu in-house, and better.

When it makes sense to build vs. buy here

Buy if:

• Speed to value matters MOST, more than customization and flexibility
• We need something working tomorrow, not weeks from now
• Maintenance capability is limited (as it is right now)

Build if:

• We have concerns over data privacy
• We have the capacity to dedicate 6-8 weeks to the project
• Our docs workflow proves too complicated for Promptless/Dosu
• We've scaled so much that SaaS pricing is monsterous

But there's one more solution: hybrid. This might look like:

• Buy Promptless or Dosu for PR drafting, Slack integration
• Build screenshot monitoring workflow if we don't go with a vendor that has it (Dosu for example)
• Own templates, linting, Vale rules, site ergonomics (as we should)

A hybrid approach can get us speed where it matters most. And a hybrid approach at this point, is my recommendation. Let's keep our costs low with whatever vendor we go with, and build the capabilities that extend it into a more powerful ecosystem ourselves.

[daniloc]

Thanks for being so rigorous about this, especially around evaluating the possible failure modes. I'd really love for this to work, and it's so easy to get bamboozled too.

What’s on the roadmap for more docs testing/docs maintenance features? Specifically for code snippet validation (syntax, functionality, etc.) and SDK/API references?

this is such an important question!

as an aside, it might be worth doing this even if we were 50/50 on our optimism, just from a learning perspective. Assuming it didn't ruin our lives. Learning opportunities here that stand out to me:

• value of custom model training for content generation (getting super specific about how they do this so we understand what we've learned would also help)
• how to scaffold automatic validation of content

[gewenyu99]

as an aside, it might be worth doing this even if we were 50/50 on our optimism, just from a learning perspective

This is a good attitude, and to maybe try in a hackathon to see if we can just do a better job ourselves ;) I suspect with our learnings from building so many LLM powered things, if this doesn't work out we can make it work lol

[daniloc]

maybe we should rent a runpod for a few days and train Nemotron 3 Nano against our docs and blog posts either way...

[sarahxsanders]

TIL Inkeep has a new Docs Writer agent that turns product updates into docs, but there's not a ton of info. It seems very new. Adding as an option to explore nonetheless because it's a platform we are already using, it just requires us reaching out to Inkeep

[sarahxsanders]

Closing with decision: We are going to proceed with a pilot with Inkeep's Content Writer, see more information here