The headless information structure is an natural emergence of the separation of information storage, administration, optimization, and entry from the providers that write, course of, and question it. With this structure, you may handle your information from a single logical location, together with permissions, schema evolution, and desk optimizations. And, to high it off, it makes regulatory compliance lots less complicated, as a result of your information resides in a single place, as a substitute of being copied round to each processing engine that wants it.
We name it a “headless” information structure due to its similarity to a “headless server,” the place you must use your individual monitor and keyboard to log in. If you wish to course of or question your information in a headless information structure, you’ll have to convey your individual processing or querying “head” and plug it into the information — for instance, Trino, Presto, Apache Flink, or Apache Spark.
A headless information structure can embody a number of information codecs, with information streams and tables as the 2 commonest. Streams present low-latency entry to incremental information, whereas tables present environment friendly bulk-query capabilities. Collectively, they provide the flexibility to decide on the format that’s most fitted to your use circumstances, whether or not it’s operational, analytical, or someplace in between.
First, let’s check out streaming within the headless information structure.
