Find themes in a small batch of feedback

The problem

You've got 20, 50, maybe 100 free-text responses from a survey or feedback form. You could read them all, but it would take a while and you might miss patterns. You need to summarise what people are saying without spending hours on it.

The solution

Paste your responses directly into Claude or ChatGPT and ask it to identify themes. For small batches, this is quick, free (or cheap), and surprisingly effective. You can have a conversation: ask for themes, then drill into specific ones, then pull out quotes for your report. Always verify AI-generated themes against your original data - AI can occasionally misinterpret responses or create themes that don't quite match what people said.

What you get

A list of 5-10 themes with the rough percentage of responses in each, plus 2-3 representative quotes per theme. You can ask follow-up questions to explore specific themes or get quotes that would work well in a particular context (for funders, for the board, etc.).

Before you start

• Your feedback in a document or spreadsheet (10-100 responses works best)
• A Claude or ChatGPT account (free tier is fine for small batches)
• An idea of what you want to learn from the feedback

When to use this

• You've got 10-100 feedback responses to summarise
• You need a quick thematic summary for a report or meeting
• You want to explore what people are saying before deciding on a more structured analysis
• The feedback is from a one-off survey or event, not an ongoing stream

When not to use this

• You've got more than 100-150 responses. The AI will struggle to hold it all in context and may miss things. Use the 'at scale' approach instead
• You need to process feedback regularly. Set up the programmatic approach once and reuse it
• The feedback contains sensitive information you can't share with a third party
• You need a defensible, auditable methodology for formal evaluation

Steps

1. 1

   Check the feedback for sensitive content

   Skim through and check for anything you shouldn't upload: names, health details, safeguarding concerns, contact information. Either remove these manually or consider whether this feedback needs human-only analysis. For most routine feedback, you're fine.

2. 2

   Format your feedback simply

   Put each response on its own line, or number them (1. Response one, 2. Response two). Remove completely blank responses. You don't need to clean up spelling or grammar. The AI handles messy text well.

3. 3

   Start with a broad request

   Paste your feedback into Claude or ChatGPT and try: "Here are [X] responses from our beneficiary feedback survey. Please identify the main themes, estimate roughly what percentage of responses mention each theme, and give me 2-3 example quotes for each theme."

4. 4

   Refine the themes

   Review what it gives you. Are any themes too broad? Ask it to break them down. Are some too similar? Ask it to merge them. Try: "The 'communication' theme seems broad. Can you break that into more specific sub-themes?"

5. 5

   Explore sentiment if useful

   If you want to know the tone, ask: "For each theme, what's the general sentiment - positive, negative, mixed?" or "Which themes have the most negative feedback?" This helps prioritise what to address.

6. 6

   Pull quotes for your report

   Ask for quotes tailored to your audience: "Give me the most compelling quote for each theme that would resonate with funders" or "Which quotes would work well in our annual report?" The AI can find quotes that illustrate your points effectively.

7. 7

   Verify the themes against original responses

   AI can occasionally misinterpret responses or group things oddly. Spot-check by reading 5-10 original responses and checking they match the themes assigned. If something seems off, ask the AI to reconsider or do the categorisation yourself for those responses.

8. 8

   Note down what you did(optional)

   Write a sentence or two about your method: "Thematic analysis of 67 beneficiary feedback responses using Claude, December 2024, with manual verification of themes." This helps when someone asks how you did the analysis.

Tools

Resources

Part of this pathway