标签：#analytics

If you’ve ever struggled with CALCULATE() or wondered why SUMX() behaves differently from SUM() , this guide is for you. DAX (Data Analysis Expressions) is the language that powers Power BI , Analysis Services , and Power Pivot — enabling dynamic calculations, filtering, and time intelligence. Below is a categorized cheat sheet of essential DAX functions , plus examples showing how to use each in real-world Power BI scenarios. Filtering & Context These functions control how filters are applied and evaluated in your calculations. Function Example Description CALCULATE() CALCULATE(SUM(Sales[Amount]), Region[Name] = "Nairobi") Changes filter context to calculate total sales for Nairobi. FILTER() FILTER(Sales, Sales[Amount] > 10000) Returns a table filtered by condition. ALL() CALCULATE(SUM(Sales[Amount]), ALL(Region)) Ignores filters on Region. REMOVEFILTERS() CALCULATE(SUM(Sales[Amount]), REMOVEFILTERS(Region)) Removes filters from Region. VALUES() VALUES(Customer[City]) Returns unique list of cities. SELECTEDVALUE() SELECTEDVALUE(Product[Category], "All") Returns selected category or “All” if none. TREATAS() TREATAS(VALUES(Temp[City]), Customer[City]) Applies one table’s values as filters on another. KEEPFILTERS() CALCULATE(SUM(Sales[Amount]), KEEPFILTERS(Product[Category] = "Electronics")) Keeps existing filters and adds new ones. ALLSELECTED() CALCULATE(SUM(Sales[Amount]), ALLSELECTED(Region)) Respects user selections in visuals. ALLEXCEPT() CALCULATE(SUM(Sales[Amount]), ALLEXCEPT(Sales, Sales[Year])) Removes all filters except Year. Aggregation Summarize or aggregate data across rows or columns. Function Example Description SUM() SUM(Sales[Amount]) Adds all sales amounts. AVERAGE() AVERAGE(Sales[Amount]) Calculates mean value. COUNT() COUNT(Customer[ID]) Counts non-blank entries. COUNTROWS() COUNTROWS(Sales) Counts rows in a table. DISTINCTCOUNT() DISTINCTCOUNT(Customer[ID]) Counts unique customers. MIN() MIN(Sales[Amount]) Finds smallest sale. MAX() MAX(Sales[Amo

Matomo is an open-source web analytics platform that keeps full ownership of visitor data on infrastructure you control, a privacy-first alternative to Google Analytics. This guide deploys Matomo using Docker Compose with a MariaDB backend, Nginx as the entry point, and Certbot issuing the TLS certificate before the stack goes live. By the end, you'll have Matomo tracking a website with a signed HTTPS certificate at your domain. Prepare Docker $ sudo usermod -aG docker $USER $ newgrp docker Set Up the Project 1. Create the project directory: $ mkdir matomo && cd matomo 2. Open the firewall: $ sudo ufw allow 80/tcp $ sudo ufw allow 443/tcp 3. Create the environment file: $ nano .env MYSQL_ROOT_PASSWORD = your_strong_mysql_root_password MYSQL_PASSWORD = your_strong_mysql_matomo_password Use passwords of at least 16 characters. 4. Create the Nginx config: $ mkdir nginx $ nano nginx/matomo.conf server { listen 80 ; server_name matomo.example.com ; location /.well-known/acme-challenge/ { root /var/www/certbot ; } location / { return 301 https:// $host$request_uri ; } } server { listen 443 ssl http2 ; server_name matomo.example.com ; ssl_certificate /etc/letsencrypt/live/matomo.example.com/fullchain.pem ; ssl_certificate_key /etc/letsencrypt/live/matomo.example.com/privkey.pem ; location / { proxy_pass http://app:80 ; proxy_set_header Host $host ; proxy_set_header X-Real-IP $remote_addr ; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for ; proxy_set_header X-Forwarded-Proto $scheme ; } } Replace every matomo.example.com occurrence with your domain — it appears in both server blocks and the certificate paths. Deploy with Docker Compose $ nano docker-compose.yaml services : db : image : mariadb:11.4 command : --max-allowed-packet=64MB restart : always volumes : - matomo-db-data:/var/lib/mysql environment : - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} - MYSQL_DATABASE=matomo - MYSQL_USER=matomo - MYSQL_PASSWORD=${MYSQL_PASSWORD} networks : - matomo-net app : image

Plausible Analytics is an open-source, self-hosted web analytics tool that tracks traffic without cookies or personal data collection, a privacy-first alternative to Google Analytics. This guide deploys Plausible Community Edition using Docker Compose, fronts it with Nginx, and secures it with a Let's Encrypt certificate. By the end, you'll have Plausible tracking a website's traffic securely at your domain. Configure the Environment 1. Clone the Community Edition repo: $ mkdir ~/plausible $ cd ~/plausible $ git clone https://github.com/plausible/community-edition.git $ cd community-edition 2. Generate a secret key: $ openssl rand -base64 64 | tr -d '\n' 3. Create the environment file: $ nano .env ADMIN_USER_EMAIL = admin@example.com ADMIN_USER_NAME = admin ADMIN_USER_PWD = ADMIN_PASSWORD BASE_URL = https://plausible.example.com SECRET_KEY_BASE = YOUR_SECRET_KEY_BASE DATABASE_URL = postgres://postgres:postgres@plausible_db:5432/plausible_db CLICKHOUSE_DATABASE_URL = http://plausible_events_db:8123/plausible_events_db Fill in your email, a strong admin password, your domain, and the secret key generated above. 4. Expose the app port via a Compose override (auto-merged with compose.yml ): $ nano compose.override.yaml services : plausible : ports : - 127.0.0.1:8000:8000 Deploy with Docker Compose $ docker compose up -d $ docker compose ps Confirm the app, Postgres, and ClickHouse (events DB) containers are all running. Front with Nginx and Let's Encrypt 1. Install Nginx: $ sudo apt update $ sudo apt install nginx -y 2. Create the virtual host: $ sudo nano /etc/nginx/sites-available/plausible.conf server { listen 80 ; listen [::]:80 ; server_name plausible.example.com ; access_log /var/log/nginx/plausible.access.log ; error_log /var/log/nginx/plausible.error.log ; location / { proxy_pass http://127.0.0.1:8000 ; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for ; proxy_http_version 1.1 ; proxy_set_header Upgrade $http_upgrade ; proxy_set_header Connection "Upgr

I'm launching SmartTaxCalc.in on Product Hunt on Tuesday, 14 July 2026 . It's a 38-tool Blazor WebAssembly tax + finance calculator I've written about here before ( the SEO/schema saga , and dropping mobile LCP from 6-8s to under 2s ). The Product Hunt launch algorithm heavily rewards products that arrive with a real coming-soon follower base — day-of upvotes correlate strongly with pre-launch "Notify me" clicks. My PH page started with 1 follower . I had 9 days to get to 50+. The obvious answer: post on LinkedIn, ask friends, DM your network. All of that has ceilings (you can only ask a favor once). The non-obvious answer that has no ceiling: convert your own organic search traffic into PH followers automatically. This is the ~100 lines of Blazor code that does that, plus the design decisions I made along the way. It's also self-cleaning — after the launch date, the banner disappears with no manual work required. Steal the pattern for your own launch. The problem SmartTaxCalc gets modest but real organic traffic — mostly from Google Search Console impressions on tax-season queries. That traffic is the warmest possible audience for a PH launch (they already found the site, they're in the target demo). But how do you route them to a PH page without: Disrupting the tax content (they came for a tax calculator, not a marketing pitch) Cannibalizing the existing tax-season banner (which drives users to /tax-calendar/ — a real retention lever) Leaving code debt after 14 July (a dead PH banner still on the site in September) Losing the dismiss preference across page navigations (SPA reality — no page refresh) Those constraints ruled out a modal, a full-width interrupt, and a "hardcoded remove after launch" approach. The design Slim horizontal bar at the top of every page. Sits ABOVE the existing tax-season banner. PH-brand orange, different from the tax-season banner's yellow/red so both are visually distinguishable when stacked. Dismissible per-user via localStorage . Auto

Introduction I used to think "connecting to a database" was one skill. Turns out it's two: connecting to a database chilling quietly on your own laptop, and connecting to one living in the cloud, behind a login, in this case, an SSL certificate that will not let you in until you treat it with respect. This week I did both. Same tool (Power BI), same dataset, two very different vibes. Grab a coffee, here's the full walkthrough local PostgreSQL first, then Aiven's cloud version, side by side, screenshots and all. Part 1: Local PostgreSQL → Power BI Step 1 : Create a schema Nothing fancy, just giving my table a home: CREATE SCHEMA powerbi ; Step 2 : Import the dataset Right-click the new schema → Import Data in DBeaver, point it at your CSV, and let the wizard do its thing. Step 3 : Check the table landed properly A quick peek at the columns to make sure nothing got mangled on the way in. Step 4 : Connect Power BI In Power BI Desktop: Get Data → Database → PostgreSQL database. In the Server field, type localhost (or 127.0.0.1 ) and your database name. localhost Choose Import , hit OK, and log in with your local username and password. Click Load . That's it. That's the whole local experience. Part 2: Aiven PostgreSQL (Cloud) → Power BI Now for the part that actually taught me something. Step 1 : Grab your connection details Everything you need lives on Aiven's Overview page: Host, Port, Database name, User, SSL mode. Your service URI will look something like this (don't worry, this isn't a real password, Aiven masks it in the console): postgres : // avnadmin : •••••••• @ pg - xxxxxxxx - yourproject . c . aivencloud . com : 22016 / defaultdb ? sslmode = require Step 2 : Import the dataset into Aiven Same DBeaver wizard as before, just pointed at the Aiven connection instead of local. CREATE SCHEMA powerbi ; Step 3 : Aiven's certificate. Download the CA cert from the Overview page: Now here's the part that actually tripped me up: Power BI's PostgreSQL connector doesn't ha

Data Modeling, Relationships, and Schemas in Data Analytics In the fields of data analytics, data warehousing, and database management, modeling and schema design are the fundamental pillars used to organize and query information efficiently. This article provides a comprehensive guide to these core concepts. 1. Data Modeling Data modeling is the architectural process of designing how data is stored, interconnected, and accessed within a system. Core Questions Addressed: Storage: What specific data points need to be captured? Structure: How should individual tables be organized? Connectivity: How do these tables interact with one another? Levels of Data Models: Conceptual Model: A high-level business perspective focusing on entities and their relationships, devoid of technical specifications. Logical Model: Defines specific attributes, keys, and relationships. It is independent of the Database Management System (DBMS). Physical Model: The actual implementation within a database, including technical details like indexes, partitions, and storage requirements. 2. Relationships Relationships define the logic of how data in one table corresponds to data in another. One-to-One (1:1): A single record in Table A relates to exactly one record in Table B. One-to-Many (1:M): The most common relationship; for example, one Customer can place many Orders . Many-to-Many (M:M): Multiple records in one table relate to multiple records in another. This requires a Junction Table (Bridge Table) to function. Example: One Student can enroll in many Courses, and one Course contains many Students. 3. SQL Joins Joins are used to combine rows from two or more tables based on a related column. Join Type Description Inner Join Returns only the records that have matching values in both tables. Left Join Returns all records from the left table and the matched records from the right. Right Join Returns all records from the right table and the matched records from the left. Full Outer Join Returns

INTRODUCTION When I started using Power BI, I only thought of visuals like charts and graphs. However, as I progressed, I discovered a great data dashboard is built on great data models. Data Modelling is the process of organizing your data tables and defining how they relate to each other so Power BI can combine them into meaningful reports and dashboards. Good, designed data makes it easier and faster to maintain. Why is data Modelling Important Well-organized data makes it easier to manage data. Reduction of the duplicates. Ensures data consistency. Understanding Relationships Relationships allow the data table to give communication using fields. For example, Customer Table stores all information about a customer. Product Table store product details Sales Table stores all information about the transactions. Power BI connects the information between the customer’s name and Customer Id rather than repeating them it connects the information using joins. Going through relationships I discovered schemes. Scheme is the way tables are organized in databases. There are different types of schemes e.g. Star Schema, snowflake schema and Flat table. Star Schema A star schema is a data model with one central fact table and dimension table surrounding it. Fact table A table that stores events, transactions of what happened. • Total sales • average sales • quantity sold Dimension table A dimension table describes the items in the fact table. The table contains descriptive information. • The customer table describes the customer • How much sales were made The fact table sits in the center, while the dimension tables surround it—forming a star. Dashboard designs A good dashboard has to fit one page. A dashboard should show critical information. Update automatically when data changes. Focus on data understanding and decision making. Conclusion Power BI taught me that a great report are built from a a great dashboard which is achieved by having great models. Structuring a data into

Target built a generative AI system to improve marketing campaign forecasting by retrieving and ranking similar historical campaigns. Using embeddings, vector search, and LLM ranking, it replaces rule-based workflows. Evaluation shows 75% top-1 and 100% top-3 coverage. The system reduces manual effort, improves consistency, and uses feedback loops to refine retrieval using campaign outcomes. By Leela Kumili

I log commutes in a spreadsheet because mobility apps smooth over the ugly legs. Last week I added a column I should have tracked years ago: carry seconds ? time from curb to platform when stairs replace ramps. The hidden leg My one-wheel leg is fine on paper. Three metro exits on my route have no elevator during maintenance. Carrying a 14 kg wheel down 22 stairs does not show up in trip duration. It shows up in whether I arrive annoyed enough to skip coffee. What I logged (one week) Exit Stairs Carry time (s) Mood after (1-5) North gate 22 38 2 Side ramp (control) 0 8 4 East stairs 16 29 3 Battery delta on those days? Within noise. Mood delta? Not noise. A cheap decision rule I turned this into a go/no-go check before leaving: if stairs > 15 AND carry_weight_kg > 12: prefer transit-only or locker elif stairs > 0 AND wet_floor: walk the wheel (no riding in station) else: ride It is blunt. It works better than pretending every leg is rideable. Assumptions up front Wheel weight includes pads and charger pouch (~14 kg for my commuter setup). I am not timing competitive carries ? just whether I can do this daily without hating it. Your threshold differs if every exit has elevators. What I would do differently I would log carry seconds from day one, same tab as distance and battery percent. Range math without carry math is incomplete for anyone who mixes metro and one-wheel. I work around personal EVs and sometimes cross-check specs on the official Kingsong catalog. https://www.kingsong.com/collections/electric-unicycle

The clock reads 6:47 in the third quarter. Your team is down three points, facing 4th and 2 at midfield. For decades, the conventional wisdom was automatic: punt the ball away and hope your defense makes a stop. But on Sunday across America's NFL stadiums, something remarkable is happening. Teams are challenging that wisdom with data, and the results are reshaping how football is played. In the 2023 NFL season, teams went for it on 4th down approximately 23% more often than they did a decade earlier. Some of this increase stems from rule changes and philosophical shifts, but the real driver is analytics. Teams now have access to comprehensive data showing that going for it on 4th down is dramatically undervalued by traditional football thinking. The margin between analytical expectation and actual performance reveals one of the most significant inefficiencies in professional sports. This article explores the data patterns behind 4th down decision-making, revealing why teams willing to challenge convention are winning more games than Vegas expects, and what the numbers tell us about the future of NFL strategy. The NFL Data Ecosystem: More Information Than Ever Understanding NFL analytics requires first appreciating the sheer volume of data available to modern front offices. We're not talking about simple box scores anymore. Teams now collect: Tracking data : Real-time positioning of all 22 players on every play, collected at 10 frames per second Biometric data : Player fatigue levels, GPS tracking during games, heart rate variability, and recovery metrics Situational data : Down and distance, field position, score differential, time remaining, and opponent tendencies Personnel data : Matchup analysis comparing specific offensive and defensive units Environmental data : Weather conditions, field surface characteristics, altitude, and crowd noise levels This data ecosystem emerged gradually. NFL teams began serious analytics initiatives in the early 2010s, largely insp

Introduction When a query runs in ClickHouse®, the database does much more than simply read data and return results. Before execution begins, ClickHouse® parses the SQL statement, analyzes it, applies optimizations, and builds an execution plan that determines the most efficient way to process the query. Understanding query execution plans is one of the most valuable skills for anyone working with ClickHouse®. They provide visibility into how queries are executed, helping you identify bottlenecks, validate optimization efforts, and troubleshoot performance issues. In this article, we'll explore how ClickHouse® generates execution plans, the different EXPLAIN modes, and how to interpret them for better query optimization. Why Query Execution Plans Matter A SQL query defines what data you want, but it doesn't explain how the database retrieves it. Consider the following query: SELECT country , count () FROM events GROUP BY country ; Although the query looks simple, ClickHouse® must determine: Which data parts to read Whether primary indexes can reduce the scan If data skipping indexes can be used How aggregation should be performed Whether parallel execution is possible How intermediate results should be merged A query execution plan provides answers to these questions, making it an essential tool for performance tuning. The ClickHouse Query Lifecycle Every query passes through several stages before producing results. The lifecycle typically looks like this: SQL Query │ ▼ Parser │ ▼ Analyzer │ ▼ Optimizer │ ▼ Query Plan │ ▼ Execution Pipeline │ ▼ Results Each stage plays an important role: Parser validates SQL syntax. Analyzer resolves tables, columns, and expressions. Optimizer applies query optimizations. Query Plan determines the logical execution steps. Pipeline distributes work across multiple threads. Execution processes the data and returns the results. Understanding this workflow makes execution plans much easier to interpret. Introducing the EXPLAIN Statement

Alina Krasavina explains how Delivery Hero successfully deprecated Google Analytics and migrated to an internal user tracking platform. She discusses how a simplistic, highly scalable architecture allowed them to handle 10 times more load while capturing 97% of tracking data. By Alina Krasavina

Here's something that'll keep you up at night: 67% of World Cup 2026 goals in the 85th+ minute came from teams that were losing at the time . That's significantly higher than the 43% rate we saw in the 70-80 minute window. This single statistic reveals a hidden pattern in how desperation fundamentally rewires attacking strategy when the clock ticks down to the final whistle. As someone who's spent the last three months drowning in World Cup 2026 broadcast data, match statistics, and possession metrics, I've become obsessed with understanding how pressure affects team behavior in those nail-biting final minutes. The conventional wisdom says that late-game goals are chaotic, desperate, and unpredictable. But the data tells a much more interesting story—one about tactical discipline collapsing under psychological weight. The Numbers Behind the Drama Let me walk you through what we found when analyzing 64 matches from the 2026 tournament across 16 days of group stages. Time Period Total Goals Avg. Pass Completion % Shots on Target Defensive Errors 0-30 min 24 82.3% 18 3 30-60 min 31 81.7% 26 5 60-75 min 28 79.4% 24 8 75-85 min 19 76.8% 22 12 85-90 min 18 71.2% 19 18 90+ min (stoppage) 14 68.9% 16 22 Notice the decline? By the 85-90 minute window, pass completion drops to 71.2%—that's an 11-point deterioration from the opening 30 minutes. But here's where it gets weird: defensive errors triple in that same window. Teams aren't just playing sloppily; they're making genuinely catastrophic mistakes. Team-Specific Patterns: The Pressure Responders Not all teams crack under late-game pressure equally. Here's where the real story emerges: Team 85+ Min Goals Scored 85+ Min Goals Conceded Goal Differential Win Rate (Tight Matches) Argentina 6 2 +4 85% France 5 3 +2 72% Brazil 7 4 +3 81% England 3 5 -2 58% USA 4 6 -2 62% Morocco 5 2 +3 79% Japan 2 7 -5 41% What jumps out immediately? Argentina and Brazil are outliers . They scored 13 combined goals in the final 5 minutes but conc

Whether you're brand new to Power BI or just getting started with data analytics, this guide walks you through everything you need to know about data modeling — from how tables connect, to the schemas that make your reports fast and reliable. What Is Data Modeling in Power BI? Imagine you have three spreadsheets: one with your customers , one with your products , and one with your sales transactions . Individually, each table tells you something. But together, they can tell you which customer bought which product, when, and for how much . That's exactly what data modeling is: the process of organizing your data tables and defining how they relate to each other so Power BI can combine them into meaningful reports and dashboards. A data model in Power BI has three core building blocks: Tables — your data sources (Excel files, databases, CSVs, cloud services, etc.) Relationships — the links between tables that tell Power BI how data connects Measures & Calculations — formulas (in DAX) that compute totals, averages, and other insights A well-designed data model is the difference between a report that loads in seconds and one that takes forever. It's also what keeps your numbers accurate and your dashboards easy to use. Why Does Data Modeling Matter? Here's a simple analogy: a city without roads is just a collection of buildings. Data modeling is the road system that lets you travel between your tables. Without a good data model: Your visuals may show incorrect or duplicated numbers Filters in one chart won't affect another Reports will be slow and hard to maintain With a good data model: Clicking on a customer in one visual automatically filters every other visual Calculations are accurate and consistent You can easily add new data sources without rebuilding everything Types of Tables: Facts vs. Dimensions Before diving into schemas and relationships, you need to understand the two types of tables that make up most Power BI models. Fact Tables A Fact table stores the ev

Anthropic recently reported that Claude now handles around 95% of its internal analytics requests, letting employees query business data independently instead of relying on data teams. The company attributes this result less to advances in models and more to data governance, semantic definitions, and operational discipline. By Renato Losio

You can't optimize what you don't measure. Every blog post about conversion optimization, A/B testing, or paid ads assumes you have reliable tracking in place. But most developers set up analytics as an afterthought — dropping a script on the page and calling it done. The result is data that's incomplete, untrustworthy, and ultimately useless for making decisions. This guide gives you a developer-first approach to conversion tracking. We'll cover event instrumentation, attribution setup, funnel visualization, and the specific tracking architecture you need to answer real business questions. No marketing jargon. No vague advice. Just the exact setup that turns your analytics from a vanity dashboard into a decision-making tool. The Tracking Mindset Before you write any code, understand what you're trying to learn. Tracking every possible event creates noise. Tracking the wrong events leads to wrong conclusions. Start with one question: "What are the 3-5 actions a user takes between discovering my product and paying me money?" Map these actions in order. That's your funnel. Every event you track should map directly to a step in that funnel. For a typical SaaS product, the funnel looks like this: Discovery: User visits your site from a traffic source Engagement: User reads content, explores features, or uses a tool Intent: User clicks "Sign Up" or "Start Trial" Conversion: User completes signup and activates Revenue: User upgrades to a paid plan If you track these five steps reliably, you can answer 90% of the marketing questions that matter: Which traffic source brings the most valuable users? Where do users drop off? What's my true cost per acquisition? Event Instrumentation: What to Track and How Events are the atomic unit of conversion tracking. An event is any action a user takes that you want to measure. Let's build your event taxonomy from the ground up. Foundational Events (Track These First) These four events are non-negotiable. Set them up before you do anythi

ClickHouse HTTP API: A Complete Beginner's Guide Introduction When most people think about interacting with a database, they usually imagine connecting through a database client or application. However, ClickHouse also provides a simple and powerful HTTP API that allows you to query and manage your database using standard HTTP requests. The ClickHouse HTTP API provides a universal interface for communicating with your ClickHouse server. Since almost every programming language and automation tool supports HTTP, it becomes an excellent choice for integrations, monitoring, scripting, and lightweight applications. In this guide, you'll learn what the ClickHouse HTTP API is, why it's useful, and how to perform common database operations using simple HTTP requests. What Is the ClickHouse HTTP API? The ClickHouse HTTP API is a built-in interface that enables clients to communicate with a ClickHouse server using the HTTP protocol. Instead of connecting through the native TCP protocol, you simply send HTTP requests and receive responses in formats such as JSON, CSV, TSV, XML, or plain text. The HTTP interface is: Language agnostic Easy to integrate with web applications Firewall friendly Simple to test using tools like cURL, Postman, or a web browser Because of its simplicity, the HTTP API is widely used for automation, dashboards, data pipelines, and monitoring systems. Why Use the HTTP API? The ClickHouse HTTP API offers several advantages: No dedicated database driver is required. Works with virtually every programming language. Easy integration with REST-based applications. Supports multiple output formats. Ideal for automation and scripting. Perfect for cloud-native applications and microservices. Common Operations Using the HTTP API, you can: Execute SQL queries Insert data Create and modify tables Retrieve query results Export data in different formats Automate database operations Authentication Options ClickHouse supports multiple authentication methods when using th

You type "k8s deployment troubleshooting" into your documentation search. The top result is a page about Kubernetes architecture that never mentions the word "troubleshooting." It is exactly what you need. BM25 would have missed it entirely. This is the promise of vector search: finding documents by meaning, not just matching words. In 2025 and 2026, vector search has moved from niche ML engineering to a core Elasticsearch capability. If you are building search for AI applications - RAG pipelines, semantic Q&A, recommendation systems - understanding how Elasticsearch handles vectors is no longer optional. I have spent the past year building RAG pipelines at Cloudera, and I have learned that vector search is powerful but easy to misuse. This post covers what works, what does not, and how to implement it in production. Why Vector Search Matters (And When It Does Not) BM25, which we covered in a previous post, is brilliant at matching exact terms. But it is fundamentally lexical. It does not understand that: "k8s" and "kubernetes" are the same thing "docker container" and "containerization" are related concepts "out of memory error" and "heap exhaustion" describe the same problem Vector search solves this by converting text into high-dimensional numerical vectors (embeddings) where semantically similar content lives close together in vector space. A query for "k8s deployment troubleshooting" gets embedded into a vector, and Elasticsearch finds the nearest document vectors - even if they do not share a single keyword. But vector search is not a replacement for BM25. It is a complement. BM25 is faster, requires no ML infrastructure, and excels at exact-term matching. Vector search is slower, requires embedding models, and shines at conceptual similarity. The best search systems in 2026 use both. How Elasticsearch Stores and Indexes Vectors Elasticsearch introduced the dense_vector field type in version 7.x and has dramatically improved it through 8.x and into 2026. Here

This article presents a straightforward approach to automatically and efficiently tune hyperparameters for machine learning models using Optuna as the optimisation framework. We explore how to use both Optuna’s native storage options and InterSystems IRIS as a database backend to track the progress of hyperparameter searches. We also show how MLflow can be used to monitor experiments and manage models through its tracking and model registry UI. This article is based on this Kaggle Notebook , which you can run and directly edit yourself. When training ML models, the choice of hyperparameters can strongly influence performance. They are not the only factor, but they can significantly affect both convergence and generalisation. Tuning hyperparameters manually takes a lot of effort. This is especially true because hyperparameters interact with each other, so tuning them independently is usually not enough. For example, higher regularisation may require a lower learning rate for more stable optimization. A more complex model may require stronger regularization to avoid overfitting, but at the same time, a very small learning rate on a complex model can make learning too slow. Optuna is an MIT-licensed open source library, which allows commercial use, that automates hyperparameter search for ML models developed with the most popular frameworks such as scikit-learn, PyTorch, TensorFlow, and LightGBM. It works by defining a search space and an objective metric to either minimize or maximize. Optuna then explores the search space efficiently to find well-performing configurations. Here we use Optuna to tune a LightGBM model on a dummy dataset and show how to scale the search using shared database storage. We will also use MLflow for experiment tracking and model registry, and IRIS DB as a possible Optuna storage backend for concurrent studies. We will use the California Housing dataset, commonly used in ML examples, to populate IRIS tables and run the tuning workflow. Note:

There's a line usually pinned on the Roman philosopher Seneca: luck is what happens when preparation meets opportunity. People put it all over social media and like most things on social media, it gets repeated so often that it stops meaning anything. So let me try to make it mean something again, with a math equation and a football match that happened recently at the latest FIFA World Cup 2026. The equation nobody writes down We talk about luck like it's a single mysterious force, either you have it or you don't. But it's not one thing. It's two things multiplied together: Luck = Preparation × Opportunities Look at what that multiplication does. If your preparation is zero, it doesn't matter how many opportunities show up, zero times anything is still zero. And if you're the most prepared person alive but you never put yourself in front of a single opportunity, same result. Zero. The lucky people aren't the ones who got more luck handed to them. They're the ones who kept both numbers high. They got good and they kept showing up to the table where things happen. Hold that thought. Let's go to Texas. Japan, the Netherlands, and the 88th minute On June 14th, 2026, Japan played the Netherlands in their World Cup group opener in Arlington, Texas. On paper it was a mismatch in the most literal, physical sense. The Netherlands are tall . Van Dijk, Van de Ven, the whole spine of that team is built like a row of wardrobes. Japan are one of the shorter sides in world football, quick, technical, but not the people you'd bet on to win a header. If you were designing a contest specifically to humiliate the Japanese, you'd make it about jumping. And for most of the night, the script ran exactly as the bodies predicted. The Dutch dominated the run of play, around 60% possession, more passes, more touches in the box, the better expected goals. Van Dijk, a defender, rose for a cross and headed the Netherlands ahead. Later Summerville restored their lead. The Oranje even won the aer