ch04
dz / designing_data_intensive_applications / ch04Summary
Chapter 4: Encoding and Evolution
Node Tree
- actor_model
- data_representation_programs
- encoding
- modes_of_dataflow
- schema_evolution
Nodes
data_representation_programs | |
content | Data representation in programs |
children | byte_sequence, bytes_in_mem, in_mem_bytes, in_memory |
in_memory | |
content | in-memory |
children | optimized_for_cpu |
parents | data_representation_programs |
byte_sequence | |
content | Self-contained sequence of bytes |
parents | data_representation_programs |
optimized_for_cpu | |
content | Optimized for CPU |
parents | in_memory |
in_mem_bytes | |
content | in-memory to bytes |
parents | encoding, data_representation_programs |
bytes_in_mem | |
content | Byte sequence to in-memory |
parents | data_representation_programs |
encoding | |
content | encoding |
children | avro, built_in_language, in_mem_bytes, standardized_encodings |
built_in_language | |
content | built-in language libraries |
parents | encoding |
remarks | avoid this approach |
standardized_encodings | |
content | Standardized encodings |
children | CSV, JSON, XML, binary, text_based, thrift_protobuf |
parents | encoding |
JSON | |
content | JSON |
children | msgpack, text_based |
parents | standardized_encodings |
XML | |
content | XML |
parents | standardized_encodings |
CSV | |
content | CSV |
children | text_based |
parents | standardized_encodings |
text_based | |
content | text-based |
parents | CSV, JSON, standardized_encodings |
binary | |
content | Binary |
parents | thrift_protobuf, msgpack, standardized_encodings |
msgpack | |
content | msgpack, BJSON, etc |
children | binary |
parents | JSON |
schema_evolution | |
content | Schema evolution |
children | compatibility, databases, relationship_encode_decode, schema, vs_schemaless |
thrift_protobuf | |
content | Thrift and Protocol Buffers |
children | binary, rpc_protobuf, schema, tag_field |
parents | standardized_encodings |
schema | |
content | Schema |
parents | schema_evolution, thrift_protobuf |
tag_field | |
content | Tag field |
children | unique_tag_number |
parents | thrift_protobuf |
compatibility | |
content | Compatibility |
children | backward, forward, only_add_remove_default, unique_tag_number |
parents | schema_evolution |
forward | |
content | Forward |
parents | compatibility |
backward | |
content | Backward |
parents | compatibility |
unique_tag_number | |
content | Unique Tag Number |
parents | compatibility, tag_field |
avro | |
content | Avro |
children | decode_exact_schema, no_tag_numbers, object_container, only_add_remove_default, readers_schema, writers_schema |
parents | encoding |
decode_exact_schema | |
content | decoded using exact same schema |
parents | avro |
writers_schema | |
content | Writer's Schema |
children | encode_data, resolves_differences |
parents | avro |
readers_schema | |
content | Reader's schema |
children | decode_data, resolves_differences |
parents | avro |
decode_data | |
content | decode data |
children | encode_data (semantic pair), resolves_differences |
parents | readers_schema |
resolves_differences | |
content | Resolves differences between read and write schema on decode |
parents | writers_schema, decode_data, readers_schema |
encode_data | |
content | Encode Data |
parents | writers_schema, decode_data |
only_add_remove_default | |
content | only add/remove fields with default value |
parents | avro, compatibility |
object_container | |
content | Object Container File |
children | self_describing |
parents | avro |
no_tag_numbers | |
content | No tag numbers |
children | dynamic_generated_schema |
parents | avro |
dynamic_generated_schema | |
content | Dynamically generated schemas |
parents | no_tag_numbers |
self_describing | |
content | Self-describing |
children | apache_pig |
parents | object_container |
apache_pig | |
content | Apache Pig |
parents | self_describing |
relationship_encode_decode | |
content | Relationship betwen process that encodes the data and the process that decodes the data |
parents | schema_evolution |
vs_schemaless | |
content | Allows same flexibility as schema-less/schema-on-read, also with better guarantees about data and tooling |
parents | schema_evolution |
databases | |
content | Databases |
children | data_outlives_code, migrating, unknown_fields |
parents | schema_evolution, modes_of_dataflow |
modes_of_dataflow | |
content | modes of dataflow |
children | databases, services |
data_outlives_code | |
content | Data Outlives Code |
parents | databases |
migrating | |
content | Migrating |
parents | databases |
unknown_fields | |
content | Unknown Fields |
parents | databases |
services | |
content | Services |
children | API_exposed_by_server (description), REST, RPC, clients, servers, web_service |
parents | modes_of_dataflow |
RPC | |
content | RPC |
children | async_message_passing, compat_encoding_scheme, futures_promises, grpc, indempotence, make_request_look_like_func (description), problems, remote_call_procedure (accronym), rpc_protobuf, service_discovery |
parents | services |
REST | |
content | REST |
children | design_principles_of_http (description) |
parents | services, web_service |
clients | |
content | Clients |
children | app_network_requests (example), web_browswer (example) |
parents | services |
web_browswer | |
content | Web Browser |
parents | clients |
app_network_requests | |
content | Application making network requests |
parents | clients |
servers | |
content | Servers |
children | API_exposed_by_server, clients_to_other_service |
parents | services |
API_exposed_by_server | |
content | API expsoed by server |
parents | services, servers |
clients_to_other_service | |
content | Can be clients to another service |
children | service_oriented_arch |
parents | servers |
service_oriented_arch | |
content | Service-oriented architecture |
children | independent_deployable_easier_maintain, microservices_arch (rebranded as) |
parents | clients_to_other_service |
microservices_arch | |
content | Microservices Architecture |
parents | service_oriented_arch |
independent_deployable_easier_maintain | |
content | Services designed to be independently deployable/evolvable in order to make applications easier to change and maintain |
parents | service_oriented_arch |
web_service | |
content | Web Service |
children | REST, SOAP, http_comm_proto |
parents | services |
http_comm_proto | |
content | HTTP used as underlying communication protocol |
parents | web_service |
SOAP | |
content | SOAP |
children | WDSL, xml_based |
parents | web_service |
design_principles_of_http | |
content | Design philosophy built on principles of HTTP |
parents | REST |
xml_based | |
content | XML based |
children | web_services_description_language |
parents | SOAP |
WDSL | |
content | WDSL |
parents | SOAP, web_services_description_language |
web_services_description_language | |
content | Web Services Description Language |
children | WDSL (abbreviation) |
parents | xml_based |
problems | |
content | Problems |
children | timeout, unpredictable |
parents | RPC |
timeout | |
content | timeout |
parents | problems |
unpredictable | |
content | unpredictable |
parents | problems |
remote_call_procedure | |
content | Remote Call Procedure |
parents | RPC |
grpc | |
content | gRPC |
children | streams |
parents | RPC |
make_request_look_like_func | |
content | tries to make request to remote service look like a function/method call in a programming language |
children | location_transparency (term to describe this) |
parents | RPC |
rpc_protobuf | |
content | RPC using protobuf |
parents | thrift_protobuf, RPC |
location_transparency | |
content | Location Transparency |
parents | make_request_look_like_func |
indempotence | |
content | Indepotence |
children | deduplication_mechanism |
parents | RPC |
deduplication_mechanism | |
content | Mechanism for Deduplication |
parents | indempotence |
futures_promises | |
content | futures/promises |
children | encapsulate_async_actions (description) |
parents | RPC |
encapsulate_async_actions | |
content | Abstractions that encapsulate asynchronous actions that may fail |
parents | futures_promises |
service_discovery | |
content | Service Discovery |
parents | RPC |
compat_encoding_scheme | |
content | Backward/forward compatibility inhereted from underlying encoding scheme |
parents | RPC |
streams | |
content | streams |
children | series_req_resp (description) |
parents | grpc |
series_req_resp | |
content | series of requests and responses over time |
parents | streams |
async_message_passing | |
content | Asynchronous message-passing systems |
children | btwn_rpc_db, message, message_broker |
parents | RPC |
message_broker | |
content | Message Broker |
children | message_queue (AKA) |
parents | async_message_passing |
message | |
content | message |
children | consumers_subscribers (delivered to), queue_topic (sent to) |
parents | async_message_passing |
btwn_rpc_db | |
content | Between RPC and database |
parents | async_message_passing |
consumers_subscribers | |
content | Consumers/Subscribers |
parents | message |
queue_topic | |
content | Queue/Topic |
parents | message |
message_queue | |
content | Message Queue / msg-oriented middleware |
parents | message_broker |
actor_model | |
content | Actor Model |
children | actor, actor_model_desc (description), distributed_actor_framework |
actor_model_desc | |
content | Programming model for concurrency in single process |
parents | actor_model |
actor | |
content | actor |
children | actor_comm_async_message |
parents | actor_model |
distributed_actor_framework | |
content | Distributed Actor Framework |
children | scale_app_across_multi_nodes (description) |
parents | actor_model |
scale_app_across_multi_nodes | |
content | used to scale application across multiple nodes |
parents | distributed_actor_framework |
actor_comm_async_message | |
content | Communicates with other actors by sending and receiving messages asynchronously |
parents | actor |