Natural Language Processing (NLP)
The branch of AI focused on enabling computers to understand, interpret, and generate human language, encompassing tasks from text classification to machine translation.
Natural language processing is the field that makes human-computer communication in everyday language possible. It spans a wide range of tasks: tokenization, part-of-speech tagging, named entity recognition, sentiment analysis, machine translation, summarization, question answering, and open-ended text generation. Before LLMs, each task typically required a separate specialized model.
The field has undergone a dramatic transformation. Pre-2018 NLP relied heavily on handcrafted features, rule-based systems, and task-specific architectures. The introduction of BERT (2018) and GPT (2018-2023) shifted the paradigm to pre-trained transformer models that could handle virtually any text task through fine-tuning or prompting. Modern NLP is increasingly synonymous with using LLMs.
For product teams, NLP capabilities are now accessible through API calls rather than requiring specialized ML teams. Common growth applications include automated content tagging and categorization, customer feedback analysis at scale, intelligent search with query understanding, chatbots and virtual assistants, and automated content generation. The practical skill is knowing which NLP capability to apply to each product problem and how to evaluate quality for your specific use case.
Related Terms
RAG (Retrieval-Augmented Generation)
A technique that grounds LLM responses in external data by retrieving relevant documents at query time and injecting them into the prompt context.
Embeddings
Dense vector representations of text, images, or other data that capture semantic meaning in a high-dimensional space, enabling similarity search and clustering.
Vector Database
A specialized database optimized for storing, indexing, and querying high-dimensional vector embeddings with sub-millisecond similarity search.
LLM (Large Language Model)
A neural network trained on massive text corpora that can generate, understand, and transform natural language for tasks like summarization, classification, and conversation.
Fine-Tuning
The process of further training a pre-trained LLM on a domain-specific dataset to specialize its behavior, style, or knowledge for a particular task.
Prompt Engineering
The practice of designing and iterating on LLM input instructions to reliably produce desired outputs for a specific task.