9. AI and marketing work

Evaluate the workflow, not the promise

The useful question is not whether AI is good at marketing in general. It is whether a defined workflow helps this organisation produce an acceptable result for a particular task, after checking, correction and oversight. A fast first draft can be a poor workflow if someone spends longer finding subtle errors than they would have spent writing the original.

Generative systems produce text, images or other outputs from patterns and context. Predictive systems estimate an outcome or category from inputs. Both can fail, but they require different evaluation. A generated content brief needs factual and editorial checks; a predicted donor response needs an appropriate dataset, validation and analysis of who may be misclassified. This module does not require building a predictive model or using personal data.

Define the task boundary

Choose a narrow, low-risk task such as drafting an internal service explanation from approved fictional facts. Write the input, allowed evidence, prohibited claims, output format and acceptance criteria before starting. If one route receives a better brief or extra evidence, a comparison of outputs will not isolate the workflow difference you intended to examine.

Separate drafting from judgement. A model can suggest alternatives, but the accountable person decides whether the evidence supports them and whether the output is suitable. Do not let a system grade its own factual reliability as the only quality check. An explanation of why an answer sounds plausible is not independent verification.

A task boundary also needs a failure definition. Inventing a response guarantee, exposing personal information or changing eligibility should be a consequential failure, not a small style penalty averaged away by fluent prose. Specify which errors prevent use, which can be corrected and who approves the correction. The safest response to some tasks is to keep them manual.

Grounding narrows inputs, not responsibility

Providing approved source material can reduce ambiguity, but it does not guarantee fidelity. The output can still omit a limit, combine facts incorrectly or add an unsupported inference. Ask the reviewer to trace each factual clause to the approved input and mark anything else as a proposal or error.

For this course, the approved facts are fictional. Do not upload linked readings, internal documents or client records to obtain a more impressive result. Access to a document does not settle reuse rights, organisational permission or suitability for a third-party system. If those questions are unresolved, use the supplied artefacts and manual checks.

Context should include the audience and decision, not merely background volume. A short verified brief may be more useful than a large bundle of conflicting material. Record the exact version used, especially if you compare runs. Without a stable input, another learner cannot reproduce the exercise or identify why an output changed.

Compare against a meaningful human baseline

Use the same acceptance criteria for a manual draft and an assisted draft. Count preparation, drafting, checking, correction and final review time. If the person learns from the manual task and then performs the assisted one, order effects may favour the second route. If possible, counterbalance task order across matched examples; if studying alone, acknowledge that limitation rather than inventing an experimental design you did not run.

The supplied timing dataset is fictional and exists for arithmetic practice. Manual drafting takes 25 minutes, checking 10 and revision 5: 40 minutes total. Assisted preparation and drafting take 8 minutes, checking 20 and revision 15: 43 minutes total. The first output is faster, but the whole assisted workflow is slower by three minutes in this scenario. These are not measured tool results.

Keep quality and time separate before making a joint decision. A faster result that fails a mandatory claim check is not an acceptable saving. A slower workflow may still be useful if it improves a relevant quality dimension, but the evidence must show that improvement. Avoid a single composite score that conceals a critical failure.

Test where success might stop

A workflow can perform well on familiar wording and fail when asked for an unsupported claim or when the input is ambiguous. Include a straightforward task, an ambiguity case and a prohibited-claim case. For each, inspect whether the output asks for clarification, preserves uncertainty or invents an answer. Do not evaluate only the best-looking response.

The linked research on uneven AI performance provides a reason to examine task boundaries, not a guarantee about your organisation. Appraise who performed the study tasks, what tools and evaluation were used, and which outcomes were observed. The blueprint's publication description and source details remain unverified here. If you cannot access the paper, use the original internal arguments and state the resulting evidence limitation.

Reproducibility does not mean expecting identical wording from a live model. Record the inputs, tool identity if used, date, settings available to you, outputs and evaluation method. Another person should be able to understand the comparison and apply the same checks. A model update or changed task can invalidate earlier evidence; record a review trigger.

Decide whether to adopt, restrict or reject

An adoption decision should specify the allowed task, approved inputs, mandatory checks, owner, cost/time assumption and stop condition. 'Use AI for marketing' is not an operational policy. 'Use optional drafting support for internal explanations from approved fictional-style facts, with clause-level checking before use' is narrower and more reviewable.

The no-AI route is equally valid. You can compare the supplied manual and simulated assisted artefacts, calculate their fictional total times, evaluate quality and recommend a workflow without logging into any service. Label the evaluation as an analysis of authored teaching materials, not a trial of a live model. Honest provenance is itself part of the result.

Your portfolio should report what happened, what the evidence supports and where it does not transfer. Include rejected outputs and correction work, not just a polished final draft. The aim is an accountable decision about a workflow, including the possibility that a manual route is better for this task.

Use a failure ledger, not only an average score

For each output, record the error, its source, consequence, detection method and correction time. An unsupported adjective and an invented eligibility rule should not receive the same treatment simply because both are factual deviations. The first may be an editorial correction; the second could send someone to an unsuitable service. Define consequence categories before comparing the routes so that a pleasing output does not receive a more forgiving standard.

Inspect omissions as well as additions. A draft may contain no false sentence yet omit a crucial limit, such as appointment-only access. Ask whether a reader would form an unsupported expectation from the whole output. Clause-level tracing is necessary but not sufficient: the reviewer must check whether the explanation remains complete for its intended decision.

When reporting time, separate measured work from a supplied teaching scenario. If you time your own manual revision, that is your observed session, not a measurement of a model. If you analyse the fictional 40- and 43-minute totals, report exactly that provenance. Do not combine them into a fabricated experiment. If a live tool is optionally tested, save the rejected outputs too; choosing only the best run conceals variability and checking burden. An adoption recommendation should explain which failures are tolerated internally, which require correction and which make the task unsuitable. This is more informative than saying a system scored highly overall while leaving the most consequential error unexamined.

Study allocation and completion

Allocate 1.5 hours to this lesson and worked example, 1.5 to critical reading, 2 to the lab, 3 to the assignment, 1 to critique and 1 to reflection and revision: 10 hours in total. These are planning allocations, not measured duration or a timed assessment. Read in short sessions if helpful. All resources are plain text; a notebook, calculator or optional free spreadsheet is sufficient. No video, AI account or paid service is needed.

For critical reading, use the linked selection if accessible and the internal reading comparison. Record its question, argument, evidence type, assumptions and one reason it might not transfer to your organisation. A missing external reading does not prevent completion: compare the two original internal positions and state that you have not appraised the external source. Keep a source log with title, URL, access date if accessed, claim supported and limitation. Links are references, not permission to copy or upload readings to AI.

Complete the lab manually first. The supplied simulated AI output is authored teaching material, not a transcript, real research or verified model performance. Mark each claim supported, unsupported or contradicted. Optional live AI may generate alternatives from these fictional inputs only; record tool, date, instructions, checks and revisions. Never supply personal, confidential or unauthorised material. If not using AI, compare the supplied output against your manual work and label that route accurately.

For critique, use the rubric and annotated example, alone or with a consenting peer; no peer is required. For reflection, revise one consequential decision and record the old choice, new choice, evidence and unresolved uncertainty. Keep work locally. Self-completion records learning activity, not professional competence or an independently assessed award. Version: 1 November 2026; report suspected errors through the course host's available contact route, or retain a correction note if no route exists.

Practical lab — 2 hours

Use the supplied paired artefacts and fictional times. Mark every factual clause against approved inputs. Calculate whole-workflow time and explain why first-draft speed is insufficient. Add an ambiguity case asking for an unknown response deadline and a prohibited case asking for a guaranteed outcome. Manually write acceptable refusals/clarifications. Optional live testing is not required and cannot use confidential material.

Portfolio assignment — 3 hours

Produce a reproducible comparison, checked artefacts and a 1,000-word evaluation, worth 20% of the final portfolio. Include input version, task, criteria, route used, timings and their provenance, quality failures, cost assumptions, checking burden, order effects, transfer limits and an adoption/stop decision. Supplied-data analysis is a complete route but must not be described as measured live-model performance.

Critique — 1 hour

Use the rubric and example, alone or with a consenting peer. Challenge the denominator, the strongest unsupported assumption and the most consequential operational risk.

Reflection and revision — 1 hour

Revise one decision and retain both versions with reasons. State the evidence that would reverse your choice. All work can remain local; no live AI, spend, contact or publication is needed.

Self-review weights: evidence and source appraisal 25%; critical reasoning 25%; contextual fit 20%; feasibility and evaluation 20%; clarity and reflective judgement 10%. Score each 0–4, then sum(weight × score / 4). These are illustrative learning criteria, not validated academic grades.

Evidence: preserve whether times and outputs were supplied or measured. Reasoning: include checking/correction and task-boundary failures. Context: apply organisational input restrictions. Feasibility: assign approval and stop responsibility. Reflection: report an unfavourable result honestly rather than forcing adoption.

0 is missing or misleading; 1 asserts without support; 2 supplies partial relevant support; 3 links evidence, alternatives and a feasible decision; 4 tests a serious contrary account and states a reversal condition. Correct invented evidence, privacy risks and unsupported public promises before self-completion, irrespective of the total.

Weak: 'AI takes eight minutes instead of 25, so it saves 68%.' This compares drafting only and ignores failures. Competent: 'Total fictional times are 40 manual and 43 assisted; the assisted text invents a guarantee.' Strong: 'Reject unattended use. This supplied scenario is three minutes slower overall, and the guarantee fails a mandatory criterion. A narrower drafting task could be tested later with approved inputs and clause-level checks. This does not establish that all AI workflows are slower.' Annotation: the strong conclusion limits itself to the evidence and distinguishes a future test from a completed one.