designing_data_intensive_applications/ch04

ch04

dz / designing_data_intensive_applications / ch04

Summary

Chapter 4: Encoding and Evolution

Node Tree

Nodes

data_representation_programs
content Data representation in programs
children byte_sequence, bytes_in_mem, in_mem_bytes, in_memory

in_memory
content in-memory
children optimized_for_cpu
parents data_representation_programs

byte_sequence
content Self-contained sequence of bytes
parents data_representation_programs

optimized_for_cpu
content Optimized for CPU
parents in_memory

in_mem_bytes
content in-memory to bytes
parents data_representation_programs, encoding

bytes_in_mem
content Byte sequence to in-memory
parents data_representation_programs

encoding
content encoding
children avro, built_in_language, in_mem_bytes, standardized_encodings

built_in_language
content built-in language libraries
parents encoding
remarks avoid this approach

standardized_encodings
content Standardized encodings
children CSV, JSON, XML, binary, text_based, thrift_protobuf
parents encoding

JSON
content JSON
children msgpack, text_based
parents standardized_encodings

XML
content XML
parents standardized_encodings

CSV
content CSV
children text_based
parents standardized_encodings

text_based
content text-based
parents standardized_encodings, CSV, JSON

binary
content Binary
parents standardized_encodings, thrift_protobuf, msgpack

msgpack
content msgpack, BJSON, etc
children binary
parents JSON

schema_evolution
content Schema evolution
children compatibility, databases, relationship_encode_decode, schema, vs_schemaless

thrift_protobuf
content Thrift and Protocol Buffers
children binary, rpc_protobuf, schema, tag_field
parents standardized_encodings

schema
content Schema
parents thrift_protobuf, schema_evolution

tag_field
content Tag field
children unique_tag_number
parents thrift_protobuf

compatibility
content Compatibility
children backward, forward, only_add_remove_default, unique_tag_number
parents schema_evolution

forward
content Forward
parents compatibility

backward
content Backward
parents compatibility

unique_tag_number
content Unique Tag Number
parents tag_field, compatibility

avro
content Avro
children decode_exact_schema, no_tag_numbers, object_container, only_add_remove_default, readers_schema, writers_schema
parents encoding

decode_exact_schema
content decoded using exact same schema
parents avro

writers_schema
content Writer's Schema
children encode_data, resolves_differences
parents avro

readers_schema
content Reader's schema
children decode_data, resolves_differences
parents avro

decode_data
content decode data
children encode_data (semantic pair), resolves_differences
parents readers_schema

resolves_differences
content Resolves differences between read and write schema on decode
parents writers_schema, readers_schema, decode_data

encode_data
content Encode Data
parents decode_data, writers_schema

only_add_remove_default
content only add/remove fields with default value
parents compatibility, avro

object_container
content Object Container File
children self_describing
parents avro

no_tag_numbers
content No tag numbers
children dynamic_generated_schema
parents avro

dynamic_generated_schema
content Dynamically generated schemas
parents no_tag_numbers

self_describing
content Self-describing
children apache_pig
parents object_container

apache_pig
content Apache Pig
parents self_describing

relationship_encode_decode
content Relationship betwen process that encodes the data and the process that decodes the data
parents schema_evolution

vs_schemaless
content Allows same flexibility as schema-less/schema-on-read, also with better guarantees about data and tooling
parents schema_evolution

databases
content Databases
children data_outlives_code, migrating, unknown_fields
parents modes_of_dataflow, schema_evolution

modes_of_dataflow
content modes of dataflow
children databases, services

data_outlives_code
content Data Outlives Code
parents databases

migrating
content Migrating
parents databases

unknown_fields
content Unknown Fields
parents databases

services
content Services
children API_exposed_by_server (description), REST, RPC, clients, servers, web_service
parents modes_of_dataflow

RPC
content RPC
children async_message_passing, compat_encoding_scheme, futures_promises, grpc, indempotence, make_request_look_like_func (description), problems, remote_call_procedure (accronym), rpc_protobuf, service_discovery
parents services

REST
content REST
children design_principles_of_http (description)
parents web_service, services

clients
content Clients
children app_network_requests (example), web_browswer (example)
parents services

web_browswer
content Web Browser
parents clients

app_network_requests
content Application making network requests
parents clients

servers
content Servers
children API_exposed_by_server, clients_to_other_service
parents services

API_exposed_by_server
content API expsoed by server
parents servers, services

clients_to_other_service
content Can be clients to another service
children service_oriented_arch
parents servers

service_oriented_arch
content Service-oriented architecture
children independent_deployable_easier_maintain, microservices_arch (rebranded as)
parents clients_to_other_service

microservices_arch
content Microservices Architecture
parents service_oriented_arch

independent_deployable_easier_maintain
content Services designed to be independently deployable/evolvable in order to make applications easier to change and maintain
parents service_oriented_arch

web_service
content Web Service
children REST, SOAP, http_comm_proto
parents services

http_comm_proto
content HTTP used as underlying communication protocol
parents web_service

SOAP
content SOAP
children WDSL, xml_based
parents web_service

design_principles_of_http
content Design philosophy built on principles of HTTP
parents REST

xml_based
content XML based
children web_services_description_language
parents SOAP

WDSL
content WDSL
parents web_services_description_language, SOAP

web_services_description_language
content Web Services Description Language
children WDSL (abbreviation)
parents xml_based

problems
content Problems
children timeout, unpredictable
parents RPC

timeout
content timeout
parents problems

unpredictable
content unpredictable
parents problems

remote_call_procedure
content Remote Call Procedure
parents RPC

grpc
content gRPC
children streams
parents RPC

make_request_look_like_func
content tries to make request to remote service look like a function/method call in a programming language
children location_transparency (term to describe this)
parents RPC

rpc_protobuf
content RPC using protobuf
parents RPC, thrift_protobuf

location_transparency
content Location Transparency
parents make_request_look_like_func

indempotence
content Indepotence
children deduplication_mechanism
parents RPC

deduplication_mechanism
content Mechanism for Deduplication
parents indempotence

futures_promises
content futures/promises
children encapsulate_async_actions (description)
parents RPC

encapsulate_async_actions
content Abstractions that encapsulate asynchronous actions that may fail
parents futures_promises

service_discovery
content Service Discovery
parents RPC

compat_encoding_scheme
content Backward/forward compatibility inhereted from underlying encoding scheme
parents RPC

streams
content streams
children series_req_resp (description)
parents grpc

series_req_resp
content series of requests and responses over time
parents streams

async_message_passing
content Asynchronous message-passing systems
children btwn_rpc_db, message, message_broker
parents RPC

message_broker
content Message Broker
children message_queue (AKA)
parents async_message_passing

message
content message
children consumers_subscribers (delivered to), queue_topic (sent to)
parents async_message_passing

btwn_rpc_db
content Between RPC and database
parents async_message_passing

consumers_subscribers
content Consumers/Subscribers
parents message

queue_topic
content Queue/Topic
parents message

message_queue
content Message Queue / msg-oriented middleware
parents message_broker

actor_model
content Actor Model
children actor, actor_model_desc (description), distributed_actor_framework

actor_model_desc
content Programming model for concurrency in single process
parents actor_model

actor
content actor
children actor_comm_async_message
parents actor_model

distributed_actor_framework
content Distributed Actor Framework
children scale_app_across_multi_nodes (description)
parents actor_model

scale_app_across_multi_nodes
content used to scale application across multiple nodes
parents distributed_actor_framework

actor_comm_async_message
content Communicates with other actors by sending and receiving messages asynchronously
parents actor