Latest AI and tech news
Part 1: Databricks RAG Architecture and Fundamentals...
A few years ago, solid SQL, Python, and the ability to build reliable ETL pipelines made you valuable. Today tools like GitHub Copilot…...
Apache Spark provides several configurations to control application execution, optimize data processing, and improve performance…...
The overlooked data engineering problem behind changing historical metrics — and how to build an audit-friendly, two-clock data pipeline…...
The modern data platform is no longer just about storing information and running SQL queries....
Part I | Part II | Part III | Part IV | Part V | Part VI | Part VII | Part VIII | Part IX | Part X | Part XI | Part XII | Part XIII | Part XIV | Part XV | Part XVI | Part XVII | Part XVIII | Part XIX...
But Data Engineering forced me to ask a different question:...
Many underwriting leaders do not want a black box making decisions their underwriters are accountable for.
The AI model itself is no longer the differentiator it once was. Enterprises increasingly have access to the same powerful foundation models. What will separate the leaders is something far more valuable: the context of their business. Customer relat...
Large database tables can contain millions of records, but we don’t always need to retrieve every row to inspect data, test queries, or…...
1. Introduction: Understanding the Data Storage Challenge...
AI Functions in dynamic Iceberg tables: pay the model only for changed rows...
In 1986, statisticians Trevor Hastie and Robert Tibshirani recognized a fundamental problem in data modeling. Standard linear regression…...
If you’re curious where the future is going, check out my review of DuckDB 2.0 and Polars 2.0/2.1 … DuckDB is moving towards a more Client/Server architecture … Polars is just fixing core issues and moving towards a default streaming and spill-to-dis...
This article outlines how Delta Lake tables managed in Databricks and those managed in Snowflake can be shared in both directions without duplicating data. This is achieved through the use of shared object storage (ADLS) and open table formats. The a...
Seven CDC problems I shared with my AI team before they fed an agent. For each one: what it is, when it matters, when it does not, and a…...
Sebastian-Ion Nae has put two small neural networks on a Raspberry Pi Pico 2 W, and published the project on GitHub. One network is a world model: given the current frame and an action, it predicts the next frame and the reward. The other is an actor...
When Databricks announced that it was launching a CDP product, it created a frenzy of activity on LinkedIn. It was an unusual move that created more questions than answers. Every expert chimed in with their two cents, and few added anything beyond ho...
If you are a data scientist, data analyst, or anyone who works with big data, you have likely used SQLite, or even use it daily....
Teams usually compare Windsor.ai and Supermetrics when they want ad data in their reports without paying enterprise prices. Both start under $50 a month, and both send marketing data into the tools your team already uses....
Share your voice on how data teams are navigating the agentic era. Take the survey and you’ll get early access to the 2027 State of Analytics Engineering Report....
Skyvia and Airbyte both move data into your warehouse, but they handle the upkeep differently. Skyvia takes care of it for you. With Airbyte, you decide whether Airbyte does it or your own team does. That’s the real Skyvia vs Airbyte decision, and it...
What happens when your developers can produce code faster than the rest of your delivery process can absorb it?...
Fivetran and Skyvia both move data from one place to another. But beyond that, they serve different needs....
A practical guide to data pipeline architectures: the layers every pipeline has, the common patterns and when to use each, a decision table, and a reference design for SaaS and database data....
On October 21, 2026, our own release pipeline built, signed and published a malicious litellm==1.107.0. We published the root cause analysis on October 23....
Introduction
Run Group-Bys Faster
Run Warehouse SQL Directly
Encrypt Sensitive Columns
Other Additions in 2.1
Final Thoughts...
Businesses generate data from almost every activity, including customer interactions, online transactions, business applications…...
Last quarter's revenue was $14.6 million. Or $14.2 million. Or $14.1 million. The ledger says one thing. The Lakehouse table feeding your BI says another. The dashboard the board looked at this morning says a third....
A while back, I wrote What the Heck is GizmoSQL?, about Philip Moore's project that took DuckDB, an in-process library, and turned it into a proper server with concurrent sessions, authentication, TLS, and an Arrow Flight SQL endpoint. It solved a re...
Taiwan's StreamActivate at SWITCH 2026, Marina Bay Sands Expo & Convention Centre, Singapore, 27–29 October 2026....
Inbound datastreams are notoriously unpredictable. Whether you are ingesting raw payloads from REST API’s, webhook triggers, unstructured…...
The post Why your schema shouldn’t mirror your database (and what to do instead) appeared first on SQLServerCentral....
