Answer

Start with a recurring workflow whose inputs, intended output and owner you can describe. Check that the information may be used and the output can be reviewed. Test the smallest useful part against the current process and simpler alternatives. Do not build AI where rules or search suffice, or where uncertainty and risk cannot be managed.

Key takeaways

  • Start with a recurring problem and a clear outcome, not a model.
  • Check the evidence, access, risk and who takes responsibility when the answer is uncertain.
  • Test the smallest useful part against today's process. Choose rules, search or no new system if that is enough.

Start with the work that needs doing

A workflow is a better starting point than a list of possible AI features. Describe who does the work, what starts it and what outcome that person needs. “Help support” is too broad. “Find the right instruction for an incoming product question and draft an answer for a handler to review” is something you can investigate.

Map today’s steps first. A delay caused by missing information or unclear ownership may not need a language model.

Describe inputs, outputs and uncertainty

Write down what information is available when the work starts. Is it structured fields, free text, documents or knowledge held by people? Then define a useful output: a retrieved document, a draft, a classification or a change in another system. Each needs different checks.

Identify what the system cannot know. If a request lacks the product version, the right response may be to ask, not guess. If sources conflict, the correct output may be a handover to a person. A useful workflow must be able to express uncertainty and abstain, rather than simply produce something that looks complete.

Check data and access before choosing a model

Does someone own the information source? Can you tell whether it is current? Is the intended user allowed to read it, and may it be processed in the chosen environment? Being technically able to retrieve information does not make it appropriate for every person or purpose.

Include the information owner and the person responsible for access when defining scope. If these questions have no answers, resolving them is the next step. Choosing a larger model is not.

Hypothetical example: drafting a support answer

This example is fictional. It does not describe a customer or a delivered system.

A support handler receives a question about changing a product setting. Inputs are the question, product name and version, and approved instructions. The output should be a draft answer referring to the relevant instruction. The handler reviews and sends it. No setting is changed automatically.

First test whether better search is sufficient. If the handler can quickly find the right passage, generated text may add little. If a draft could still help, it should distinguish supported statements from missing information. A missing version or conflicting instructions should trigger a question or a handover.

An illustrative decision record might look like this:

Workflow: find instructions and propose an answer
Allowed output: draft with source
Missing product version: ask the user
Conflicting sources: refer to the responsible person
External action: handler reviews and sends

This discussion aid is not an implementation.

Choose the smallest test and a useful success criterion

Collect real examples that you are permitted to use from the workflow. Include difficult cases, missing information and situations where no answer should be given. Ask people who understand the work to explain what they would accept and why.

Compare the test with the current process and a simpler alternative. In the example, a criterion could be that the handler can verify every instruction against the correct version without repairing invented details. Also examine the work needed to review and correct the output. A quick draft that demands extensive checking may not improve the process.

Start without automatic external actions. Decide who reviews the output, who can stop the test and how to return to the previous process. Expand only when the evidence supports it.

Decision checklist: build, simplify or stop

  • Is there a recurring need with a named user and accountable owner?
  • Are inputs, permitted outputs and uncertain cases described?
  • Are current sources and permitted access available?
  • Can someone who understands the work verify the output?
  • Are the consequences of an error manageable, and can the action be reversed?
  • Is there a smaller test and a clear basis for continuing or stopping?

Do not build AI when simple rules or search meet the need, when the information cannot be used, or when nobody can judge the output. Pause if the consequences of an error cannot be managed or responsibility is unclear. A well-founded decision not to build is useful work.

Once the workflow is chosen, the next question is whether it needs a language model, search, ordinary rules or a combination. The task should determine that choice, not the appeal of a tool.

An AI production sprint can be the next step when the workflow, ownership and test are defined. Start with that decision, not a promise of automation.