The idea did not start with artificial intelligence. It began while I was looking for a question for my final Data Analytics & AI project and came across a post about the growth of independent sellers on Amazon. It mentioned that, in 2023, more than 10,000 independent sellers had surpassed $1 million in sales for the first time and that, by 2025, that figure had grown to more than 75,000; the same material also pointed to 36% growth over the previous year.
What caught my attention was not simply the size of Amazon or the magnitude of those numbers. Behind every seller that manages to grow in an environment like that is a business trying to understand its customers, its product, and the competitors fighting for the same purchase well enough to keep moving forward. Reaching a meaningful position is difficult; maintaining it when customers can compare prices, products, and experiences within seconds is probably even harder.
Amazon also offered a context that was difficult to ignore. Thousands of products compete in front of the same customer, price differences are visible, and alternatives are only a click away. Beneath each product, however, something even more interesting is happening: hundreds or thousands of people explain, in their own words, what they value, what disappointed them, what worked, what failed, and why they would—or would not—buy again.
All of that information is available. The problem begins when we stop reading ten or twenty opinions and try to understand thousands of them consistently. At that point, reviews stop being merely individual comments and begin to look like a potential source of customer and competitive intelligence.
My first idea could have remained a sentiment-analysis exercise, which would have been perfectly reasonable within a Data Analytics & AI program. The more I thought about the problem, however, the less interested I became in simply determining whether a review was positive or negative. What I wanted to understand was whether those opinions could help identify where a brand appeared to be winning, where it appeared to be losing, and how much evidence was actually available to support that conclusion.
That shift ended up changing the project. The objective stopped being about using a technology and became about building a defensible way of turning unstructured customer language into information that could provide context for a decision. The central question eventually became quite simple to state: if a brand were competing in this segment, where would it be winning and where would it be losing against its direct competitors according to what customers themselves were saying?
There was a second part to the question that would become even more important: how could I distinguish a real competitive signal from a difference that only looked interesting because it happened to appear in a particular sample?
Taking the question into a real market
To test that idea, I needed to stop talking about “products” and “competitors” in the abstract. I chose the premium over-ear active noise-cancelling headphone segment and used Sennheiser as the anchor brand, benchmarking it against Bose, Sony, Bang & Olufsen, and Bowers & Wilkins. The choice of Sennheiser also had a connection to the context in which I was developing the project: the company is headquartered in Wedemark, near Hannover, and its origins are closely tied to engineering developed in the region.
Sennheiser no fue un cliente real. Ninguna de las cinco compañías encargó, participó o fue informada del análisis; se trató originalmente de mi proyecto final del Weiterbildung y posteriormente continué refinándolo como pieza profesional de portfolio. Utilizar una empresa y un mercado reales permitía formular mejor la pregunta, pero no convertía un proyecto académico en una experiencia comercial que nunca ocurrió.
The source was Amazon Reviews 2023, the public dataset developed by McAuley Lab at UC San Diego. Starting from the Electronics category, I built a corpus focused specifically on the premium product lines of the five brands, applying product filters and a minimum price threshold of $150. After cleaning the data and removing ten strict duplicates, the final dataset contained 2,302 real reviews.
Getting to those 2,302 reviews was less straightforward than it might sound. An early version of the filters classified AKG products as if they belonged to Sennheiser, while another overly broad filter produced 1,154 matches for Sony—hardly credible if the goal was to study only its premium ANC line. The dataset spans many years of products, and a criterion that is too generic can easily mix together items that contain the right keyword but do not actually belong to the same competitive set.
The solution was to move away from broad filters and work with specific product families: Momentum for Sennheiser, QuietComfort for Bose, WH-1000X for Sony, and the corresponding lines for the other brands. It was not the most technically sophisticated part of the project, but it became one of the most important decisions. If the reality we are measuring is poorly defined, no amount of sophistication later in the process can rescue the analysis; it only allows us to be wrong with greater precision.
Before letting AI analyze thousands of opinions
Once the corpus had been built, an uncomfortable question appeared. A language model could interpret thousands of reviews much faster than I could, but speed and reliability are not the same thing. If the core of the system was going to depend on an AI model’s interpretation, I first needed some way to understand how closely that interpretation matched a human evaluation.
I selected a sample of 200 reviews, 40 per brand, and labeled them manually. Each review was evaluated for its overall sentiment and across seven product dimensions: sound quality, noise cancellation, battery, comfort, build and design, value for money, and software/connectivity. That sample became the ground truth against which the models could later be evaluated.
The labeling process itself caused the design to evolve. I had initially defined six aspects, but enough comments about Bluetooth, firmware, apps, and connectivity began appearing to make it clear that I was leaving out a meaningful part of the customer experience. Software/connectivity therefore became the seventh category, not because I needed another variable for the dashboard, but because the reviews showed that my original framework was incomplete.
I then compared two models—Llama 3.3 70B and GPT-OSS 120B—against those same 200 reviews. Before running the bake-off, I established the criteria that would determine whether a model was good enough, including error against the human labels, polarity accuracy, severe sentiment reversals, and agreement in aspect detection. Defining those rules before seeing the results was not a minor methodological detail; it reduced the risk of adjusting the standard afterward around whichever model turned out to be more convenient.
If you decide what “good enough” means only after seeing the results, you are no longer just evaluating. You also risk rationalizing the option you already want to choose.
Both models passed the mandatory criteria, and the statistical comparison between them produced a p-value of 0.757. In other words, there was not enough evidence to declare a statistical winner. That made the decision more interesting: if both models were sufficiently capable for the task, then speed, operational reliability, performance on ambiguous cases, and real execution cost also had to be considered.
Llama 3.3 70B was ultimately selected. GPT-OSS showed stronger agreement in aspect detection, but Llama was roughly three times faster per review, required fewer schema retries, and performed better in certain neutral cases. Even the cost argument had to be corrected: an early reading suggested a substantial cost difference, but once complete input and output usage were included, GPT-OSS’s actual advantage amounted to only about seven cents across all 2,302 reviews.
That correction was small in economic terms but important methodologically. The project was beginning to establish a pattern: initial decisions could change when better evidence appeared, and documenting the correction was preferable to preserving a cleaner story.
When the data contradicts a reasonable story
With the model selected, I processed all 2,302 reviews and built the brand comparisons. This was where one of the moments that came to define the project emerged, because one of my original hypotheses seemed perfectly reasonable: Sennheiser has an identity strongly associated with engineering and sound quality, while Bang & Olufsen and Bowers & Wilkins have a more visible presence around design, materials, and premium positioning.
I therefore expected to find a relatively clear division between those territories. Instead, within the sample analyzed, Sennheiser had the highest percentage of mentions related to design and comfort among the five brands. Rather than confirming that it was giving up that territory to competitors more closely associated with design, the reviews showed that its own customers talked about that dimension more often than I had expected.
It was a descriptive finding, not a causal claim or a formal statistical comparison, but it was enough to answer the hypothesis as it had originally been framed. The initial idea was reasonable. It was also wrong. And that difference—between having a reasonable hypothesis and having evidence to support it—became central to the project.
A second result was even more instructive. Sennheiser had the highest average score for sound quality, exactly the kind of result that fit the brand’s reputation and the engineering narrative surrounding the case. It was clear, intuitive, and favorable to the company I had chosen as the reference point.
That was precisely why it needed to be subjected to the same standard used for unfavorable findings. When I calculated bootstrap confidence intervals, the apparent advantage over Bose no longer held: the interval crossed zero. Sennheiser still had the highest average, but the evidence was no longer strong enough to defend a statistical advantage in sound quality.
The correct conclusion was more modest: with this sample and this level of uncertainty, there was not enough evidence to claim that Sennheiser outperformed its competitors in sound quality. The most attractive finding was also the one I had to let go.
I do not think that weakened the project. On the contrary, it was one of the moments when the analysis began to demonstrate something more useful than the ability to produce results. If rigor is applied only when the data contradicts what we want to believe, then rigor stops being a method and becomes decoration.
What did survive scrutiny
Other findings did withstand the review. In noise cancellation, Sennheiser had the lowest average score among the five brands and, more importantly, the gap versus Bose and Sony—the two strongest ANC performers in this sample—remained statistically significant. Against Bang & Olufsen and Bowers & Wilkins, however, the difference could not be supported in the same way.
The correct interpretation, therefore, was not that Sennheiser was “worse than the entire industry.” What the data supported was something more specific: in this sample, its perceived ANC performance sat roughly in the same group as Bowers & Wilkins and Bang & Olufsen, but clearly below Bose and Sony. That nuance may change the headline less than it changes the meaning, which is exactly why it matters.
Battery performance showed another signal. Sennheiser also ranked last by average score, although the difference that can be stated confidently as statistically confirmed is the comparison with Sony. Other dimensions produced interesting results—for example, value for money or software/connectivity—but they were not subjected to the same level of inferential testing and therefore need to remain descriptive findings.
“It has the highest average” does not necessarily mean “it has a demonstrated competitive advantage.” In the same way, “it ranks last” does not automatically mean “it is significantly worse than every competitor.” Learning to preserve that distinction in language became as important as producing the numbers themselves.
Finding signals that disappear inside an average
The project also included an experimental layer designed to detect potential safety anomalies that could disappear inside any aggregate score. An isolated incident involving overheating, a battery issue, or an electrical problem would be unlikely to move an overall satisfaction metric, but that does not make it irrelevant to someone analyzing the product.
The first version of the detector used keywords and produced 37 candidates. A problem quickly became obvious: perfectly normal expressions in the language of audio, comfort, or emotion were triggering the rules without representing a safety incident. The detector was finding words, but not necessarily meaning.
After tightening the rules to require more context around those terms, the candidate set fell from 37 to six. I manually reviewed each one, and three genuinely contained signals worth closer attention, while the other three were still false positives. In the public version, I decided not to identify the brands associated with those incidents because they were individual, externally unverified reviews and did not provide sufficient basis for making a general safety claim about any company.
The exercise also revealed a more important limitation. False positives are visible because the detector flags something and we can then determine that it was wrong; false negatives are different. If a review describes a real issue using language that the rules do not anticipate, it simply never appears among the candidates, and we have no way to know how many such cases may have been missed.
A signal can be too small to move an average and still be important enough to deserve attention. But detecting something does not mean we have demonstrated its frequency, its cause, or its scope.
That limitation became part of the roadmap rather than something to hide as an inconvenient weakness.
There is a distance between a finding and a decision
As the project progressed, another question became increasingly important: even if the analysis had identified the right signals, what should a company actually do with them? The distinction matters because an analysis can improve a decision without being capable of making that decision.
If the reviews reveal a consistent perception gap in ANC versus Bose and Sony, a product manager could reasonably decide that the issue deserves deeper investigation. It would be very different to conclude, based on those reviews alone, that the company should immediately redesign the hardware. A decision of that kind can be costly and difficult to reverse, and it would need to triangulate the Amazon signal with additional information such as returns, warranty data, customer-support tickets, internal surveys, or market research using a different sample.
Amazon Reviews can help indicate where to look. They cannot necessarily decide what to do.
That boundary became one of the most important parts of the work. It is easy to ask more authority from an analysis than it actually has: data can reduce uncertainty, reveal an anomaly, compare alternatives, or challenge a hypothesis, but that does not mean a single public source can replace all of the operational, financial, technical, and strategic context an organization needs for an important decision.
In that sense, the most useful outcome of the project was not producing seven rankings. It was learning to separate more precisely what the data allowed me to state, what it could reasonably suggest, and what still required more evidence.
What this project demonstrates—and what it still does not
The project ultimately covered an end-to-end process: problem definition, data exploration and collection, SQL cleaning, manual construction of the ground truth, comparative model evaluation, processing of the 2,302 reviews, statistical testing, competitive analysis, and communication through an executive report and a public dashboard. Technically, I followed the six phases of CRISP-DM and a bronze/silver/gold data architecture; Python, MySQL, language models, statistics, and Tableau were all part of the process, but none of them was the purpose of the work by itself.
There are also important limitations. Amazon is a single source, and the dataset used ends in September 2023; the ground truth was labeled by a single annotator; the total available population is unknown for four of the five brands; some product lines combine different generations; and Sennheiser has incomplete price coverage within the dataset. None of these issues automatically invalidates the analysis, but they do constrain how far its conclusions can reasonably be extended.
I also cannot yet claim that the system works across any product category. Several parts of the architecture were designed for reuse—the multi-provider AI setup, the resilience pattern, the model-evaluation protocol, the statistical layer, and the data-preparation pipeline—but the seven-aspect schema, the prompt, and the product filters belong specifically to the headphone domain. Moving into another category requires understanding what matters in that market before reusing the infrastructure.
That is why I prefer to describe the architecture as designed for reuse rather than already proven to be reusable. There is an important difference between building something with the intention that it can generalize and having demonstrated that it actually does.
That distinction also defines what the project represents today. It began as the final project of a Data Analytics & AI program and later evolved into a portfolio piece and a form of professional evidence. The possibility of turning it into a commercial service was explored, but a later assessment of the market suggested that V1 should not be commercialized as it stood; its current value lies in the work itself, the methodology, the evidence produced, and the questions it allows me to continue developing.
V2: Changing the context before expanding the system
The next iteration will continue to work with Amazon Reviews. The product category and the anchor brand will change, although I have not yet decided which ones they will be. The purpose of V2 is not to add new platforms or immediately turn the project into a universal system, but to observe what happens when the same methodological logic is applied to a different domain.
That means revisiting questions that may now appear settled. Does the validation protocol still work adequately when the relevant product dimensions change? Which parts of the architecture remain stable, and which need to be redesigned? Does the separation between finding, interpretation, and recommendation remain robust enough when customer language and market problems are different?
V2 should not prove that V1 was right. It should test which parts of V1 deserve to survive.
Two specific improvements have already been identified. The first is to replace the keyword-based alert detector with a contextual evaluation embedded in the normal analysis of each review, directly addressing one of V1’s clearest limitations. The second is to reorganize the public dashboard so that summary and detail are separated, allowing a user to recognize the main findings quickly and explore deeper only when needed.
There is also a broader ambition beyond that, but it belongs to a different stage. Once the Amazon Reviews V1/V2 line has been developed and closed to a sufficient level, I am interested in exploring whether this logic can evolve into a broader customer and competitive intelligence system capable of working with reviews from other sources, such as Google Maps, Yelp, or TripAdvisor. That would introduce new problems around data acquisition, structure, and comparability, and no architecture has yet been defined for solving them.
That possibility is a future direction, not a current capability. V2 is still Amazon Reviews; multi-platform generalization would come later and would have to earn its own evidence.
What remains after 2,302 Amazon Reviews
When I started, the question seemed to be about Amazon, sentiment analysis, and artificial intelligence. After 2,302 reviews, two models, seven dimensions, several hypotheses, and a few conclusions I had to abandon, the lesson I most want to keep has much less to do with any particular tool.
Technology made it possible to process information at a scale that would have been difficult to handle manually. Statistics helped separate some signals from noise. But neither removed the need to define the problem correctly, question a reasonable hypothesis, acknowledge a limitation, or decide that an attractive result did not have enough evidence to become a conclusion.
Data is useful when it allows us to see something we could not see before. It becomes even more valuable when it forces us to stop seeing something that we only thought was true.
Perhaps that is the part of the project most worth carrying forward. Not that I built a system that always finds answers, but that I built one capable of forcing me to reconsider my own.
This project was originally developed as the final project of a Data Analytics & AI program and was later refined as a professional portfolio piece. It uses the public Amazon Reviews 2023 dataset from McAuley Lab, UC San Diego. Sennheiser, Bose, Sony, Bang & Olufsen, and Bowers & Wilkins did not participate in or commission the analysis.
The full case study, methodology detail, and interactive dashboard for this project are available on GitHub and Tableau Public.
Comments are closed.