Tim Etler

Prototyping the Right Way with AI

April 08, 2025

Starting from a higher abstraction

Prototyping is often misunderstood, commonly executed poorly, and in many cases approached backwards. When asked to build a prototype, most engineers instinctively dive in at the implementation level and try to make a fully functional test of the solution they want to try. They end up making something akin to a mechanical prototype where you make an unrefined physical prototype to test out a product. However, when it comes to software design, this skips over many layers of abstraction that need to be considered first and jumps directly into creating a high-fidelity prototype that often overlooks other potential solutions and overinvests in one direction.

Creating a high-fidelity prototype on your first attempt invests significant time into a single approach. This first attempt often consumes the entire time budget for prototyping, which is the main reason we often see prototypes making it into production.

Anything can be a prototype

A prototype doesn’t need to be at the implementation level. It’s important to begin prototyping at higher levels of abstraction first to better explore the problem space and produce lower fidelity and lower investment prototypes that can be easily thrown away. Anything that can be experimented on or tested can be a prototype at every level of abstraction within an information system.

A system design diagram can be a prototype to see how the high-level components of different system architectures connect to each other and how they integrate with components of the existing architecture.
An RFC (Request For Comments) spec can serve as a prototype for exploring and testing ideas with team members. The name itself highlights its purpose as a feedback-driven, iterative artifact.
Interface definitions can prototype different abstractions to test out usability, complexity, and expose limitations. Simple mock implementations can be tested with developers in your actual code base to evaluate real-world usage patterns before full development.
A semi-functional interactive design can also be a prototype to test out different UX and UI approaches and be tested with real users.
An outline can serve as a prototype for an article, like the notes I took when writing this one

A fortunate aspect of higher abstraction levels is that they can be explored with broad, quick prototypes that are cheaper to build, discard, and iterate on. Higher-level prototypes also allow you to more broadly explore the problem space you’re working in and avoid closing off potential solutions prematurely, and once you do eliminate a path, it eliminates a ton of lower-level solution spaces along with it.

Prototyping faster with AI

When working with AI to help you prototype, I don’t mean telling the AI to go off and come up with prototypes for you, but rather using them as a sounding board to organize your thoughts, a research assistant, and a drafter to help you create prototypes faster and speed up your pace of iteration. LLMs have unlimited time to do work in parallel in the background and they provide output that’s easy to throw away.

Organizing your thoughts more efficiently

LLMs make for particularly effective rubber duckies that can actually respond to, and help you organize your ramblings instead of sitting there with a blank plastic stare. As opposed to inanimate objects, they can help you expand your problem solving abilities. Even when they’re woefully wrong, when correcting their mistakes, I’ve come up with solutions I wouldn’t have explored otherwise. Even bad feedback can be helpful at times. The context of a chat is also a great place to dump unorganized thoughts that you can then reference to create mind maps with the help of an LLM re-organizing your inefficient ramblings, while still maintaining a context history to pull back from. I’ve always found doing this manually to be very inefficient as I have to re-read and re-write and re-organize my notes to clarify my thinking.

Researching faster and more comprehensively

As research assistants, I find they do a good job at surfacing potential solutions for a problem more exhaustively than doing research manually. While they don’t always pick out the best or most relevant solutions, you can get them to spit out a wide breadth of solutions and also use them to help filter them based on specific criteria you give them. With the sheer scope of technologies to choose from for every possible problem, I’ve found it hard to compile a comprehensive list when researching myself and have found LLMs good at looking up that knowledge from their internal stores and from large external searches that would be very time-consuming to sort through myself. LLMs can help you explore the space more efficiently and I’ve had success finding new solutions to consider that I had a hard time surfacing myself. Even when LLMs surface inadequate solution options, analyzing the weaknesses of that option can help you refine your thinking on what strengths you actually do need.

Drafting prototypes quickly

They also have a particular weakness which I actually view as a bonus for creating prototypes. LLMs tend to pick overly simplistic naive solutions to problems. While that’s a terrible trait for creating production code, this is a great thing for prototyping, and it can help engineers that are used to working with best practices in mind let go a little and create a lower quality prototype that’s easier to throw away. I find that since most of the time we spend coding we’re trying to write high quality code that follows best practices, we’re so practiced at it that we tend to overengineer prototypes and have a hard time taking ourselves out of that mindset. There are AI tools for turning your notes and thoughts into nearly every kind of prototype at any level of abstraction, and unlike you, it will do a worse job at it, meaning you’ve successfully created a prototype you want to throw away, and saved yourself time.

There’s also the nice aspect that when using an LLM, you’re not wasting anyone’s time. I try to be respectful of the time of other engineers on my team because I know how busy they are and I want to present my questions and requests for feedback to them efficiently. Having a sounding board for your thoughts can help you clean up your requests and sometimes even help you figure things out yourself.

These are all workflows I find especially helpful for creating prototypes and creating them faster. Remember, the goal here isn’t to obsess over quality, it’s to explore the problem and solution spaces faster and then throw it all away and just taking with you the knowledge you’ve acquired.

Be careful when using AI

As helpful as they can be, LLMs have some major pitfalls you need to account for. They can’t do everything, and you should be using them for exploration, not for leading the process.

Accuracy

I have approximate knowledge of many things

-Demon Cat

First, they lie, they lie frequently, very confidently, and they have no shame. Don’t just take everything it says as fact, make sure you cross reference what they tell you. The more specific the information, the more likely it is to be incorrect, so be more careful as you explore lower-level abstractions.

Recency

I’m with it. I’m hip.

-Dr. Evil

They also fall out of date. Keep in mind the nature of the information they’re providing and how often that information changes. Also, as with accuracy, the more specific the information is, the less likely it is to be up to date.

Confirmation Biases

You’re goddamn right

-Walter White

LLMs are also very agreeable. Don’t let them make you think you’re a genius and that your first idea is amazing. Focus on your own learning and gaining knowledge in the problem space, don’t ask them to do subjective evaluations or make decisions for you.

It’s good to introduce external information into their context windows to help with alignment to improve their accuracy and update their knowledge. You should treat LLMs like you have an infinite number of interns who have an unlimited amount of time to do research. They can be useful when you ask it to help you in specific ways and for compiling information or creating rough drafts, but you still need to manage them and review their output. Don’t try to get LLMs to do your job for you, use them as tools to help you do your job better.

This is not vibe coding. The goal is to improve your understanding of the problem and solution spaces, and that requires understanding all of their outputs deeply.

How to evaluate prototypes

The purpose of a prototype is to give you some kind of deliverable that you can use to test and explore potential solutions. Think about what you are trying to learn, and what tests you can perform on your prototype at the level of abstraction it’s exploring.

Testing

Testing a prototype is not always going to necessarily be a functional test that you would get from an implementation level prototype, but can also be more conceptual tests that make sense for that level of abstraction.

A system diagram test could be trying to integrate it with a system diagram of your existing architecture.
An RFC test could be checking it with other engineers for understandability, comprehension, and exposing unanswered questions.
An interface definition test could use dummy data and dummy implementations to see how easy it is to work with in your actual source code.
A UI prototype test could be running user testing to see how well users understand the UI.

Evaluation

Think of what kinds of interfaces you are testing with your prototype. Good interfaces should be as simple to use and comprehend as possible without losing expressivity to allow you to implement solutions cleanly at the level of detail the problem requires. You need to evaluate how easy it is to use, how well it fulfills your requirements, and whether that abstraction adds any restrictions that will prevent you from fulfilling your performance goals. How you balance that will depend on the problem you are trying to solve.

With a system diagram, how well does it integrate with the existing system? How many interdependencies does it add? Can that be minimized without losing flexibility?
An RFC can help expose issues with scope and help catch scope creep early. Is the solution abstraction trying to do too much? Can it be simplified by being broken up or reducing its responsibilities?
An interface definition integration test can help you identify how easy the interface is to work with. How much boilerplate does it introduce? Does it maintain a strong source of truth? How easy is it to change? Is the abstraction encapsulating the problem at the right level? How many interdependencies does it introduce?
A UI prototype can expose areas of the UI that are confusing, overloaded, or hard to interact with. Does the user notice the details? Can they figure out how to finish their task? Are key flows hidden? Are the interface patterns easy to use?

As prototypes begin to explore lower-level higher-fidelity abstractions, they become more time-consuming to produce and more complicated to test. Evaluating and choosing solutions based on prototypes should happen from the highest level of abstraction first where prototyping is both cheaper, and selecting solutions narrows down the largest parts of the problem space.

Go build better prototypes

We all recognize the value of prototypes for helping us engineer better solutions, but historically it has been hard to build them fast enough for us to fully evaluate all the potential solutions in the relatively short amount of time we allocate to them. To me, the best use of AI in coding is for accelerating prototype iteration speed, and that increase in efficiency for generating prototypes has helped me find and try out solutions I wouldn’t have had time to explore otherwise and create better, more elegant architectures with better fitting technology choices for the problems I’m solving.