Selected work

Public reference · ML and product intelligence

Gaming Community Intelligence: Sentiment & Personas

A Databricks and PySpark workflow that converts high-volume community chat into sentiment trends and privacy-conscious aggregate research personas.

Type
ML product-intelligence workflow
Role
Architect / engineer
Status
Public reference
Focus
Sentiment, segmentation, privacy, research
DatabricksPySparkDistilBERTMLflowEmbeddingsKMeansPCALLM summaries
System map

The operating path.

A deliberately simplified architecture view. The case study below explains where the important guarantees and decisions live.

  1. 01Community chat
  2. 02Clean + redact
  3. 03Sentiment model
  4. 04Trends
  5. 05Aggregate profiles
  6. 06Personas + research Q&A

The problem

Why this system needed to exist.

Community chat contains fast, candid product feedback, but reading it message by message does not scale. Raw message counts also fail to explain which themes are changing or how different groups experience the product.

The system needed to create useful product signals without turning model-generated personas into claims about real individuals or exposing direct identifiers.

The system

How the pieces work together.

The sentiment path ingests MongoDB or Delta chat, cleans messages, fine-tunes and registers a three-class DistilBERT model, scores messages in batches, and aggregates daily and topic-level trends.

The persona path redacts light PII, uses secret-scoped pseudonymous identifiers, summarizes recent conversations, generates embeddings, clusters aggregate behavior, and produces structured persona cards and a clearly labeled research-Q&A simulation.

01

Product decisions, not vanity charts

Product and design teams used the research personas to test feature workflows and understand audience needs before launches; the work also informed feature timing and cohort analysis.

02

Privacy-conscious aggregation

The public design suppresses small clusters, removes usernames, redacts common identifiers, and labels personas as generated research artifacts rather than real people.

03

Two complementary signals

Sentiment shows what is changing over time. Personas help stakeholders reason about recurring communication patterns, preferences, and support needs.

Source boundary

The public version uses synthetic messages, removes sensitive targeting fields, and contains no original data, model artifacts, MLflow runs, stakeholder deck, or private dashboard. Personas are aggregate research tools, not ground truth about individuals.

A system like this on your roadmap?

Let’s make the hard parts explicit.

Discuss the project