AI and Verifiability: A Framework for Career Planning

I recently read an article by Jason Wei (now an AI research scientist at META, previously at OpenAI): Asymmetry of verification and verifier’s rule. It got me thinking.

He proposes this idea:

Verifier’s rule: The ease of training AI to solve a task is proportional to how verifiable the task is.

In other words, how well AI can learn a task depends on how easy it is to verify. All tasks that are solvable and easy to verify will eventually be mastered by AI.

Following this logic, if you don’t want to be replaced by AI, there are two career paths to consider as I see.

Path 1: Work on tasks that are hard to verify

Jason Wei identified 5 characteristics that make a task easy for AI to learn:

1. Objective truth - everyone agrees what good solutions are

2. Fast to verify - any given solution can be verified in a few seconds

3. Scalable to verify - many solutions can be verified simultaneously

4. Low noise - verification is tightly correlated to solution quality

5. Continuous reward - it’s easy to rank the goodness of many solutions

So we do the opposite: find work that lacks these characteristics, ideally violating several at once.

Take corporate strategy for example: what makes a good strategy is debatable, depending on values and context; it takes years to see results; each decision is unique; market conditions, luck, and macro environments introduce massive noise.

Or psychotherapy: each client has different needs with no standard answer; therapeutic effects take time to observe; it’s highly personalized and can’t be verified at scale.

Other examples include: artistic creation (painters, writers, composers), architecture, fashion design, fundamental research, venture capital, investigative journalism, etc.

I’ve noticed these jobs require taste, creativity, intuition, and understanding human nature. Experience matters too. These are humanity’s competitive advantages against AI.

Path 2: In verifiable work, shift from executor to rule-maker

If your current work fits those 5 characteristics and you’re still in an execution role, you need to think about:

1. Use AI tools to replace your execution work

Let AI handle most of the execution. Transform yourself from “someone who executes based on metrics” to “someone who defines the metrics.” This will boost your productivity and make you the go-to AI person at your company, creating more opportunities ahead.

2. If existing AI tools can’t solve your tasks well

This might be a business opportunity worth exploring. This opportunity is time-sensitive. The window might only be one or two years.

For this path, the ability to learn quickly matters more than experience, because AI is a new field and iterating fast. People who are curious about new things, have agency, ask the right questions, and can define “what makes a good answer” will go further.