Curriculum/DP Design/C3.1 Product Analysis and Evaluation

Product Analysis and Evaluation | C3.1

Guiding questionHow does product analysis and evaluation inform various stakeholders and aid in a product's future development?

Being able to say precisely identify what's wrong with something is a skill, and perhaps more rare than you'd think. Most criticism of products or processes stops at "this is annoying", which is valid, but not actionable. You may feel that a movie is boring, or a game isn't fun, or a subject is not interesting, but those feelings provide little information that can be used to improve things. Product analysis is the process of turning reactions into something specific enough to address and fix, and has been involved in nearly every improvement made to a product.

Constructive discontent is my favourite idea in this topic, partly for the name. It refers to the refusal to accept that things are simply the way they are, coupled with the discipline to explain what could be different. The tools and structure features, like SWOT, Porter's Five Forces, and reverse engineering, help to make that discontent systematic rather than random, on your own work and products designed by others. It can be challenging to find the flaws in something something you designed yourself, but developing that skill can help improve your work considerably.

Students must be able toDiscuss the purpose of a product analysis exercise and why it is important for product design development.

The main goal of product analysis and evaluation is to deeply understand how all the different parts and features of a product work together. We look at what makes a product functional (does it work well?), appealing (do people want to use or buy it?), and competitive (how does it compare to similar products?). Through this careful examination, we discover actionable insights we can use to make real improvements or develop new solutions. In simpler terms, product analysis helps you figure out: What makes this product work? Why do people use or buy it? And how does it stack up against the competition?

"Actionable insights" are specific, practical recommendations that can be implemented to improve the product. For example, a thorough analysis of an application might find that users rarely use a particular feature because they weren't aware of it, rather than avoiding it by choice. Simply looking at how many people use each feature can provide useful data, but to find actionable insights we'd have to look closer. (A specific example of this is MacOS quick actions. Do you use quick actions?)

Five analytical approaches used in product analysis:

  • Understanding user needs: We need to gain insights into how consumers feel about product effectiveness and usability. We can design the most amazing kitchen gadget, but it will be a failure if people don't actually need it for anything. We can also design something that people absolutely need, but is far too difficult to use.
  • My bottle of kombucha
    The specific bottle of kombucha referred to below.
  • Evaluating form and function: assessing materials, manufacturing techniques, and aesthetics. By now this stuff should be clear, but I'll use the glass bottle of kombucha on the table next to me as an example (Not the drink itself, though it is tasty). It's a glass bottle with a plastic lid. Why is it made from these materials? Could we use aluminium or plastic instead? Do we risk losing sales if we switch materials? Is glass perceived as a premium material which helps justify a higher cost? This bottle has a relatively 'sparse' design compared to competing products, but is sold at a lower cost. Does the design communicate that it is cheaper, or that is not as good? Is it better to explain 'probiotic cultures' or let the customer pretend they know what that means?
  • Analysing competitive positioning: benchmarking against rival offerings. Super straightforward. Galen designs a new running shoe, and we purchase shoes which will be direct competition and identify how his shoes compare.
  • Strategic analysis: examining pricing models and marketing approaches. Is it better to sell our game at a fixed price or make it free to play with a season pass and in game purchases? Can we add paid cosmetic content to our game without players revolting over our perceived greed? How can we convince people to try our game if it seems similar to games they might already think are better? What worked and didn't work for similar games?
  • Sustainability audits: improving environmental responsibility across the life cycle. Will people reuse the packaging if it's a bit stronger? How can we encourage customers to recycle the product at the end of its life instead of disposing of it? Are there materials we can swap out that will reduce the impact of producing each item?

Product analysis enables designers to learn from successes and shortcomings, reexamine problems, explore new technologies, and iterate products into more appealing, user-aligned solutions.

Students must be able toIdentify the various stakeholders, such as users, manufacturers and engineers, that will help in any future development of a product.

Bringing a wide range of stakeholders into product analysis matters because something that feels obvious and easy to one group can be a genuine obstacle for someone else. Involving multiple perspectives ensures the final solution is technically refined, aesthetically pleasing, commercially viable, and broadly accessible.

A stakeholder is any person or group that has an interest in or is affected by the product. Involving them in the analysis process helps ensure that the final solution meets their needs and expectations. You are welcome to look up the specific history of the term 'stakeholder', but it isn't terribly interesting. I encourage my students to think of it as similar to 'shareholder' or 'partner', in that both of those terms refer to people who will benefit if a company or project succeeds.

StakeholderTheir contribution to product analysis
Consumersshare real-world-use feedback, such as how well the product works and how easy or difficult it was to use. They can share usability issues, express thoughts about the value of the product, and discuss their expectations.
Product Development ManagersGather and process data from all relevant sources to assist in further product development. They can act as a bridge between different stakeholder groups.
Product DesignersTypically focus on product aesthetics, user experience, function, opportunities for innovation, and sustainability. They can also evaluate whether the product's form effectively communicates its function.
EngineersWeigh in about whether the product is possible, and whether it would be more feasibile with different materials or revisions to manufacturing.
Marketing StaffReview and analyse the market for the product, identifying trends, competion, pricing strategies, promotional opportunities, and where to sell.
Customer Support StaffListen directly to customers and provide information on recurring issues and potential areas of improvement.
Business LeadersAbove all focus on making sure the product delivers a return on investment, but also that it is in alignment with strategic goals, company values, and the approved project budget.

Students must be able toAnalyse a product using a SWOT analysis based on function, performance, usability, features and materials.

A SWOT analysis is a structured framework with four quadrants. Strengths and Weaknesses are internal (within the product or organisation); Opportunities and Threats are external (from the market, technology, or regulatory environment).

Mr. K's Steam Deck
Mr. K's Steam Deck. Who needs attribution when you can just take your own photos?
QuadrantDefinitionSteam Deck Example
Strengths (internal, positive)Valued features: aesthetics, functionality, technological superiority, brand reputation"It just works!", seamless integration with Steam library, highly rated aesthetics and ergonomics
Weaknesses (internal, negative)Limitations: resource constraints, pricing, intuitiveness, market placementAging hardware, directive to ensure device doesn't losemoney, lingering compatibility and launcher issues
Opportunities (external, positive)Market trends, new technologies, growing segmentsGrowing demand for handheld devices and portability, large potential audience of parents and 'former hardcore gamers', massive catalog of games which can be optimized and marketed
Threats (external, negative)Competitors, changing regulations, economic downturns, supply chain issuesGrowing number of stronger handheld devices, memory and storage crisis, modern games require more and more power to run

Let's imagine that we've opened a new restaurant called 'Pizza Dog' (Named after one of my first kitchen inventions) and want to do a SWOT analysis. My concept is a vegetarian pizza and hot dog restaurant, with video games and other entertainment, making it a blend of restaurant and hang-out zone.


SWOTPizza Dog Analysis
StrengthsUnique menu, without competitors; strong brand identity; high quality ingredients; fun atmosphere
WeaknessesVegetarian food is less desirable to many customers; large numbers of delivery customers who don't form the same emotional connection as dine-in patrons; most offerings are best eaten 10-15 minutes after preparation, but many deliveries take 20-30 minutes to arrive
OpportunitiesDevelop menu items that travel better; provide packaging and free items that will help build connections with delivery-only customers; hang-out space can be used for events to bring in more customers
ThreatsMenu can be copied and iterated on easily; more established competition could produce the same food at lower cost due to better deals with suppliers

Porter's Five Forces (Michael Porter, 1979) complements SWOT by analysing competitive intensity at the industry level: (1) competitive rivalry, (2) supplier power, (3) buyer power, (4) threat of new entry, and (5) threat of substitution. While SWOT analyses a specific product, Five Forces analyses the whole industry.

SWOT Matcher

Dumpalicious delivers hot dumplings by drone, straight to your window, in about six minutes. Twelve observations about the business are sitting in the bank. Drag each one into the quadrant where it belongs, or select it and then select a quadrant. Watch the axis labels: the trap in SWOT is putting an external factor in an internal quadrant.

Dumpalicious
PositiveNegative
InternalExternal
Observations

Students must be able toUse data generated from relevant testing and/or reverse engineering to identify areas of the product that require improvement and to determine how well the product meets the needs of the user.

Systematic testing and reverse engineering generate objective data that replaces guesswork in product evaluation.

Testing methods:

Test typeWhat it measuresOutput
Usability testingHow easily users can operate the product; error rates; time-on-taskIdentification of confusing interfaces, missing affordances
Performance testingWhether the product meets specified functional requirements (speed, load, accuracy)Pass/fail against specification; quantitative comparison to competitors
Materials testingTensile strength, hardness, fatigue life, corrosion resistanceMaterial suitability; predicted service life
Customer review analysisReal-world issues reported by actual users at scaleRecurring pain points; issues that lab testing misses

Reverse engineering as analysis (disassembly / teardown): Physically deconstructing a product reveals component materials, manufacturing processes, assembly sequence, and structural decisions. Used by designers to understand how a competitor solved a design problem, or to identify failure modes in their own product. (For the full reverse engineering process, see B4.1 Production Systems, objective 4.1.5.)

Benchmarking compares a product's performance, features, and design against leading competitors or industry standards. It equips designers with awareness of prevailing standards, emerging aesthetic trends, and potential avenues for market differentiation.

Product Spotlight
A disassembled game boy

Teardown: Game Boy vs. Game Gear

The handheld with the worse screen won the format war.

Read spotlight →

Students must be able toCompare competing and similar products to identify opportunities for further product improvements.

The relationship between weaknesses and opportunities is central to iterative design. A weakness in one product is an opportunity for the next version, or for a competitor. A frequent modern example is 'battery life'. So many iterative products are advertised as having better battery life than their predecessors, or 'our longest lasting battery yet', and so on. Battery life is a great feature to mention for sales because people who have older devices, especially phones, may have few other issues with their device, but almost certainly have less battery life than they did when their device was newer. This table shows the capacity of all iPhone batteries, and makes a compelling case that 'battery life' has always been identified as a weakness


Three possible decisions arising from product analysis:

DecisionWhen to choose itTriggers
Invest in improvementAddressable weaknesses; clear market opportunities; stakeholder demandSWOT shows fixable weaknesses + relevant opportunities; engineers confirm feasibility
Leave unchangedStrengths outweigh weaknesses; no compelling opportunities; stable market positionBenchmarking shows product leads competitors; no significant user complaints
DiscontinueIrreparable weaknesses; strong threats; declining market; no viable opportunitiesMultiple stakeholder groups report persistent issues; technology obsolescence confirmed

Value proposition mapping helps designers articulate what makes a product worth buying, and where a competitor's value proposition exceeds their own. This identifies specific improvement targets rather than vague dissatisfaction.

Customer reviews are an important analysis input: they reveal at scale what real users experience, including failures that only emerge after extended use, which lab testing cannot replicate.

Students must be able toIdentify areas where a product is not meeting the needs of its users.

Constructive discontent is defined as "the ability to identify, articulate and address shortcomings in an existing product without simply criticising." It is a mindset that challenges the status quo while offering pathways to improvement: criticism with a purpose.

Simple criticismConstructive discontent
Statement"This product is terrible""This product fails at X, which could be solved by Y"
FocusProblem onlyProblem + proposed solution
Effect on teamDemoralisingAction-oriented; motivates improvement
ResultComplaints without actionActionable design brief for improvement

Example: Knife Block: A designer applying constructive discontent would observe: "The knife block accumulates crumbs and bacteria at the bottom of the knife slots because it cannot be disassembled for cleaning. This is a hygiene issue that could be solved by designing a dishwasher-safe block with removable slot inserts." This becomes an actionable design opportunity, not just a complaint.

Constructive discontent drives iterative design: the cycle of observe > articulate shortcoming > propose improvement > prototype > evaluate is the engine of product evolution.

Discussion
Practise constructive discontent, right now

List three things about a product you use every day that mildly annoy you. For each one, rewrite it in the "fails at X, which could be solved by Y" format from the table above. If you can't fill in a plausible Y, you probably haven't found the real problem yet, just a symptom of it.

Then swap with a partner and try to find the weak point in each other's Y: is the proposed fix actually addressable by design, or does it depend on something the designer can't control (cost the user won't pay, a supplier that won't change, a habit the market won't drop)?

If you need help choosing a product, consider something like Moodle, Meituan, your backpack, or the elevator.

Students must be able toDiscuss the purpose of a product analysis task and why it is important for product design development.

Product analysis is not an end in itself: it is the evidence base for design decisions. The full analytical cycle connects observation to action:

  1. Observe: Test the product; gather stakeholder perspectives; disassemble and reverse engineer; collect customer reviews.
  2. Analyse: Apply SWOT; benchmark against competitors; map value propositions; identify pain points through constructive discontent.
  3. Decide: Invest in improvement, leave unchanged, or discontinue, based on evidence, not opinion.
  4. Iterate: Implement changes; test again; repeat.

Why product analysis matters for design development:

  • Prevents designing in isolation: grounds decisions in user reality rather than designer assumptions.
  • Reveals non-obvious failure modes that only emerge in extended real-world use.
  • Identifies opportunities that a competitor or market trend has created.
  • Builds stakeholder buy-in: when people are involved in analysis, they feel heard and support resulting changes.
  • Supports environmentally conscious design: analysis can identify over-engineering, excessive materials use, or poor repairability.

The goal is products that are not only technically and aesthetically refined, but also commercially viable and broadly accessible to their intended users.

Ten questions covering the learning objectives for this topic. Select one answer per question, then click "Check all answers" to see your score and the explanations.

Q1 · 3.1.1 Purpose of product analysis
The primary purpose of product analysis and evaluation is to:
Analysis asks what makes the product work, why people buy it and how it compares with rivals, and its output is actionable insight: a specific, implementable recommendation. Lower cost or faster production may follow, but they are consequences rather than the purpose.
Q2 · 3.1.2 Stakeholder involvement
Which stakeholder gathers data from every other group and turns it into a single direction for further development?
Designers contribute on aesthetics, experience and sustainability, engineers on feasibility and manufacture, and marketing on trends, pricing and competition. The development manager acts as the bridge between those specialist views, which is what stops analysis becoming a set of disconnected opinions.
Q3 · 3.1.2 Stakeholder involvement
Customer support staff are a valuable source of analysis data because they:
Support teams talk to users after purchase and at scale, which surfaces recurring pain points and the failures that only appear after extended use. Customer reviews serve the same function, and both reach problems that laboratory testing cannot reproduce.
Q4 · 3.1.3 SWOT analysis
In a SWOT analysis, a market crowded with established and emerging competitor brands is classified as a:
Threats are negative and external, arising from the market, technology or regulation rather than from the product itself. The commonest error with this framework is placing an external factor in an internal quadrant: limited capacity in a design is a weakness, while rising demand for that product category is an opportunity.
Q5 · 3.1.3 SWOT analysis
How does Porter's Five Forces differ from a SWOT analysis?
The five forces are competitive rivalry, supplier power, buyer power, the threat of new entrants and the threat of substitution. SWOT examines one product against its own strengths and its market; Five Forces asks whether the industry it sits in is worth competing in at all, so the two are complementary.
Q6 · 3.1.4 Testing & reverse engineering
A team records error rates and time on task while participants operate a new interface. This is:
Usability testing measures how easily the product can be operated, and its output is the identification of confusing interfaces and missing affordances. Performance testing checks the product against its functional specification for speed, load or accuracy, materials testing covers strength and durability, and benchmarking compares the result with competitors.
Q7 · 3.1.4 Testing & reverse engineering
Designers carry out a teardown of a competitor's product mainly to:
Disassembly reveals component materials, manufacturing processes, the assembly sequence and the structural decisions behind them, which shows how a rival solved a problem and where cost or complexity has accumulated. Applied to a designer's own product, it exposes failure modes directly.
Q8 · 3.1.5 Benchmarking & opportunities
Benchmarking shows a product leading its competitors, and no significant user complaints have been recorded. The appropriate decision is to:
Analysis leads to one of three decisions. Investment suits fixable weaknesses paired with a real market opportunity, discontinuation suits irreparable weaknesses in a declining market, and leaving a product unchanged is the right call when its strengths outweigh its weaknesses and its position is stable. Choosing on evidence rather than instinct is the point of the exercise.
Q9 · 3.1.6 Constructive discontent
Which statement is an example of constructive discontent rather than simple criticism?
Constructive discontent names the specific failure and a route to fixing it, in the form of fails at X, which could be solved by Y. That turns a reaction into an actionable design brief, whereas criticism alone identifies a problem and stops, which demoralises a team without directing it.
Q10 · 3.1.7 Product analysis outcomes
Why is product analysis described as an evidence base rather than an end in itself?
Observation gathers test data, stakeholder views and teardowns; analysis applies SWOT, benchmarking and constructive discontent; the decision to improve, leave alone or discontinue follows from that evidence; and the cycle repeats. Along the way it prevents designing in isolation, reveals failure modes that only appear in real use, and builds stakeholder support for the changes that result.
Paper 2 structured questions require extended written responses. Use the sample answers and mark scheme notes to practise and self-assess.
Question 1 · 4 marks
Explain what a SWOT analysis is and give one example for each quadrant, using a handheld games console as the product.
Show example answer

A SWOT analysis is a standard product analysis tool with four quadrants: Strengths and Weaknesses (internal), Opportunities and Threats (external). (1)

  • Strength: It works straight out of the box with the user's existing game library, and reviewers rate its ergonomics highly. (1)
  • Weakness: The hardware is ageing, and some games still have compatibility problems. (1)
  • Opportunity: Demand for portable gaming is growing, and there is a large catalogue of older games that could be optimised for the device. (1)
  • Threat: More powerful rival handhelds keep appearing, and memory and storage prices are rising. (1)

Note: any correctly classified example earns the mark. The key skill is separating internal factors, which are strengths and weaknesses, from external ones, which are opportunities and threats.

Question 2 · 6 marks
Explain why a diverse range of stakeholders is critical to the success of product analysis. Refer to at least four different stakeholder groups in your answer.
Show example answer

A diverse range of stakeholders matters because something that feels obvious and easy to one group can be a real obstacle for another. Involving several perspectives produces a wider range of ideas, makes the product more accessible, and builds support for the changes that follow, because people who were consulted feel heard. (1)

  • Consumers report how the product actually performs in daily use, how easy it was to operate and whether they think it is worth the money. This reveals usability problems that controlled testing misses. (1)
  • Engineers judge whether the design is actually possible to build, and whether different materials or manufacturing changes would make it more feasible. This prevents attractive designs that cannot be made. (1)
  • Customer support staff hear the same complaints repeatedly from real users, so they can identify recurring pain points that a survey would never surface. (1)
  • Marketing staff analyse trends, competitors, pricing and where the product should be sold, so that a well-designed product also reaches the right buyers. (1)
  • Why diversity is irreplaceable: if only engineers dominate, products may be technically sound but ugly and unmarketable. If only designers dominate, products may be aesthetically pleasing but technically flawed. Bringing the groups together produces a solution that is technically sound, well presented, commercially viable and usable by a wide range of people. (1)
Question 3 · 5 marks
Explain what "constructive discontent" means and how it benefits the product design process. Use a specific example of applying constructive discontent to an existing product.
Show example answer

Definition: constructive discontent is the ability to identify, articulate and address shortcomings in an existing product without simply criticising it. It is criticism that comes with a proposed solution. (1)

Difference from simple criticism: Simple criticism says "this is bad." Constructive discontent says "this fails at X, which could be fixed by Y." This makes it actionable for design teams. (1)

Benefits: (any 2 × 1 mark)

  • Fosters innovation: challenges existing norms and breaks out of incremental thinking. (1)
  • Supports iterative design: each observation-to-improvement cycle makes the product better. (1)
  • Builds team culture: when teams practice constructive discontent, they can criticise ideas without personal blame, enabling psychological safety and risk-taking. (1)

Example: Traditional knife block. Observation: "Knife slots accumulate crumbs and bacteria because the block cannot be disassembled for cleaning: a hygiene problem." Constructive discontent framing: "Redesign the block with removable slot inserts that are dishwasher-safe." This is a design brief, not a complaint. (1)

Question 4 · 4 marks
Explain Porter's Five Forces model and give one example from the consumer electronics (smartphone) industry for each force.
Show example answer

Porter's Five Forces (1979) analyses competitive intensity and industry attractiveness. (One mark per force correctly named and applied, max 4.)

  • Competitive rivalry: Intensity of competition among existing firms. Smartphone industry: Apple, Samsung, Xiaomi, and others compete fiercely on features, price, and ecosystem: rivalry is extremely high. (1)
  • Supplier power: Ability of suppliers to dictate terms. Smartphone: TSMC (chips) and Samsung Display (OLED panels) have significant power: switching is costly and difficult. (1)
  • Buyer power: Ability of customers to influence prices. Smartphone: consumers have high power: switching brands is easy; online reviews amplify buyer influence. (1)
  • Threat of new entry: Ease of entering the market. Smartphone: moderate threat: brands like Nothing have entered; but R&D costs, patents, and supply chains create barriers. (1)
  • Threat of substitution: Availability of alternative products. Smartphone: relatively low: no direct substitute for most users, though smartwatches replace some functions. (1)
Question 5 · 6 marks
Analyse how product analysis and evaluation can help a design team decide whether to discontinue a product, invest in its improvement, or leave it unchanged. Refer to stakeholder input, SWOT analysis, and benchmarking.
Show example answer

Product analysis provides evidence for three strategic decisions: invest in improvement, leave unchanged, or discontinue. Decisions based on evidence rather than one stakeholder's opinion produce better outcomes. (1)

Stakeholder input: Customer support staff identify recurring pain points: if the same complaint persists (e.g., battery fails after 6 months), improvement is needed; if the flaw is inherent, discontinuation may be appropriate. Engineers identify whether the core technology can be updated cost-effectively; if not, discontinuation is indicated. Marketing teams identify whether the market segment is growing (invest) or shrinking (discontinue). (1+1)

SWOT analysis: a weakness that can be fixed, paired with a matching opportunity, points to invest. A handheld console with ageing hardware, sold into a growing market for portable gaming, is a clear case for a new version. If the weaknesses cannot be fixed and the threats are strong, the same analysis points to discontinue.

Benchmarking: If competitors offer equivalent performance at 30% lower cost and cost reduction is not feasible, the decision may be discontinue. If competitors lack a feature the product has and consumers value it, leave unchanged or increase marketing investment. If competitors introduce a new technology becoming standard, invest in adding it. (1)

Conclusion: Without analysis, companies might invest in products that should be discontinued, or discontinue products that could be saved with a targeted improvement. Evidence-based decisions (combining stakeholder input, SWOT, and benchmarking) produce commercially viable outcomes. (1)

SWOT analysis, MindTools
mindtools.com/amtbj63/swot-analysis
Clear guide with templates and worked examples for each quadrant. A good starting point for practising SWOT on a product you already own.
How competitive forces shape strategy, Michael Porter (1979)
hbr.org/1979/03/how-competitive-forces-shape-strate…
The original Five Forces article in Harvard Business Review. Dense, but it is the source, and reading it is more convincing than reading a summary of it.
Porter’s five forces analysis, Wikipedia
en.wikipedia.org/wiki/Porter%27s_five_forces_analys…
The five forces with a diagram and worked industry examples. Use this first if the original article is heavy going.
Thematic analysis of qualitative data, Nielsen Norman Group
nngroup.com/articles/thematic-analysis
How to turn a pile of user reviews and interview notes into themes you can act on, using coding and affinity diagramming. The method behind 3.1.2.
The Value Proposition Canvas, Strategyzer
strategyzer.com/library/the-value-proposition-canvas
A one page tool for stating what a product does for a user and why they would pay for it. Useful for turning the weaknesses you find into an opportunity in 3.1.5.

Linking Questions

  • Which aspects of ergonomics are vital to establish when analysing the usability of products? (A1.1)
  • To what extent does the evaluation of products rely on user-centred research methods? (A2.1)
  • Why is it important to know which manufacturing techniques were used to make a product when conducting product analysis? (A4.1)
  • What types of information can designers gain from product analysis and evaluation in relation to production systems? (B4.1)
  • Why is it the responsibility of the designer to learn from product analysis and evaluation tasks when redesigning products? (C1.1)
  • What is the relationship between life-cycle analysis and product analysis? (C3.2)