diff --git a/go/README.md b/go/README.md index 6715605eb9..66a9861508 100644 --- a/go/README.md +++ b/go/README.md @@ -2,8 +2,12 @@ Fory is a blazingly fast multi-language serialization framework powered by just-in-time compilation and zero-copy. +For comprehensive documentation, see the [Fory Go Guide](https://fory.apache.org/docs/guide/go/). + ## Installation +**Requirements**: Go 1.24 or later + ```bash go get github.com/apache/fory/go/fory ``` @@ -50,93 +54,6 @@ func main() { } ``` -## Supported Types - -### Basic Data Types - -- `bool` -- `int8`, `int16`, `int32`, `int64`, `int` -- `uint8` (byte), `uint16`, `uint32`, `uint64` -- `float32`, `float64` -- `string` - -### Collection Types - -- `[]bool`, `[]int16`, `[]int32`, `[]int64` -- `[]float32`, `[]float64` -- `[]string` -- `[]any` (dynamic slice) -- `map[string]string`, `map[int]int`, `map[string]int`, and more - -### Time Types - -- `time.Time` -- `time.Duration` - -## Configuration Options - -Fory Go supports configuration through functional options: - -```go -// Enable reference tracking for circular references -f := fory.New(fory.WithTrackRef(true)) - -// Enable compatible mode for schema evolution -f := fory.New(fory.WithCompatible(true)) - -// Set maximum nesting depth -f := fory.New(fory.WithMaxDepth(20)) - -// Combine multiple options -f := fory.New( - fory.WithTrackRef(true), - fory.WithCompatible(true), -) -``` - -## Cross-Language Serialization - -Fory Go enables seamless data exchange with Java, Python, C++, Rust, and JavaScript: - -```go -// Go -f := fory.New() -f.RegisterNamedStruct(User{}, "example.User") -data, _ := f.Serialize(&User{ID: 1, Name: "Alice"}) -// 'data' can be deserialized by Java, Python, etc. -``` - -## Thread Safety - -The default Fory instance is not thread-safe. For concurrent use: - -```go -import "github.com/apache/fory/go/fory/threadsafe" - -f := threadsafe.New() - -// Safe for concurrent use -go func() { f.Serialize(value1) }() -go func() { f.Serialize(value2) }() -``` - -## Documentation - -For comprehensive documentation, see the [Fory Go Guide](https://fory.apache.org/docs/guide/go/). - -Topics covered: - -- [Configuration](https://fory.apache.org/docs/guide/go/configuration) - Options and settings -- [Basic Serialization](https://fory.apache.org/docs/guide/go/basic-serialization) - Core APIs and usage patterns -- [Type Registration](https://fory.apache.org/docs/guide/go/type-registration) - Registering types for serialization -- [Supported Types](https://fory.apache.org/docs/guide/go/supported-types) - Complete type support reference -- [References](https://fory.apache.org/docs/guide/go/references) - Circular references and shared objects -- [Schema Evolution](https://fory.apache.org/docs/guide/go/schema-evolution) - Forward/backward compatibility -- [Cross-Language](https://fory.apache.org/docs/guide/go/cross-language) - Multi-language serialization -- [Code Generation](https://fory.apache.org/docs/guide/go/codegen) - Experimental AOT code generation -- [Thread Safety](https://fory.apache.org/docs/guide/go/thread-safety) - Concurrent usage patterns -- [Troubleshooting](https://fory.apache.org/docs/guide/go/troubleshooting) - Common issues and solutions - ## How to Test ```bash diff --git a/go/fory/LICENSE b/go/fory/LICENSE new file mode 100644 index 0000000000..261eeb9e9f --- /dev/null +++ b/go/fory/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/go/fory/README.md b/go/fory/README.md new file mode 100644 index 0000000000..69aa9dc98f --- /dev/null +++ b/go/fory/README.md @@ -0,0 +1,140 @@ +# Apache Fory™ Go + +Fory is a blazingly fast multi-language serialization framework powered by just-in-time compilation and zero-copy. + +## Installation + +**Requirements**: Go 1.24 or later + +```bash +go get github.com/apache/fory/go/fory +``` + +## Quick Start + +```go +package main + +import ( + "fmt" + "github.com/apache/fory/go/fory" +) + +type User struct { + ID int64 + Name string + Age int32 +} + +func main() { + // Create a Fory instance + f := fory.New() + + // Register struct with a type ID + if err := f.RegisterStruct(User{}, 1); err != nil { + panic(err) + } + + // Serialize + user := &User{ID: 1, Name: "Alice", Age: 30} + data, err := f.Serialize(user) + if err != nil { + panic(err) + } + fmt.Printf("Serialized %d bytes\n", len(data)) + + // Deserialize + var result User + if err := f.Deserialize(data, &result); err != nil { + panic(err) + } + fmt.Printf("Deserialized: %+v\n", result) +} +``` + +## Supported Types + +### Basic Data Types + +- `bool` +- `int8`, `int16`, `int32`, `int64`, `int` +- `uint8` (byte), `uint16`, `uint32`, `uint64` +- `float32`, `float64` +- `string` + +### Collection Types + +- `[]bool`, `[]int16`, `[]int32`, `[]int64` +- `[]float32`, `[]float64` +- `[]string` +- `[]any` (dynamic slice) +- `map[string]string`, `map[int]int`, `map[string]int`, and more + +### Time Types + +- `time.Time` +- `time.Duration` + +## Configuration Options + +Fory Go supports configuration through functional options: + +```go +// Enable reference tracking for circular references +f := fory.New(fory.WithTrackRef(true)) + +// Enable compatible mode for schema evolution +f := fory.New(fory.WithCompatible(true)) + +// Set maximum nesting depth +f := fory.New(fory.WithMaxDepth(20)) + +// Combine multiple options +f := fory.New( + fory.WithTrackRef(true), + fory.WithCompatible(true), +) +``` + +## Cross-Language Serialization + +Fory Go enables seamless data exchange with Java, Python, C++, Rust, and JavaScript: + +```go +// Go +f := fory.New() +f.RegisterNamedStruct(User{}, "example.User") +data, _ := f.Serialize(&User{ID: 1, Name: "Alice"}) +// 'data' can be deserialized by Java, Python, etc. +``` + +## Thread Safety + +The default Fory instance is not thread-safe. For concurrent use: + +```go +import "github.com/apache/fory/go/fory/threadsafe" + +f := threadsafe.New() + +// Safe for concurrent use +go func() { f.Serialize(value1) }() +go func() { f.Serialize(value2) }() +``` + +## Documentation + +For comprehensive documentation, see the [Fory Go Guide](https://fory.apache.org/docs/guide/go/). + +Topics covered: + +- [Configuration](https://fory.apache.org/docs/guide/go/configuration) - Options and settings +- [Basic Serialization](https://fory.apache.org/docs/guide/go/basic-serialization) - Core APIs and usage patterns +- [Type Registration](https://fory.apache.org/docs/guide/go/type-registration) - Registering types for serialization +- [Supported Types](https://fory.apache.org/docs/guide/go/supported-types) - Complete type support reference +- [References](https://fory.apache.org/docs/guide/go/references) - Circular references and shared objects +- [Schema Evolution](https://fory.apache.org/docs/guide/go/schema-evolution) - Forward/backward compatibility +- [Cross-Language](https://fory.apache.org/docs/guide/go/cross-language) - Multi-language serialization +- [Code Generation](https://fory.apache.org/docs/guide/go/codegen) - Experimental AOT code generation +- [Thread Safety](https://fory.apache.org/docs/guide/go/thread-safety) - Concurrent usage patterns +- [Troubleshooting](https://fory.apache.org/docs/guide/go/troubleshooting) - Common issues and solutions diff --git a/go/fory/doc.go b/go/fory/doc.go new file mode 100644 index 0000000000..c06788c116 --- /dev/null +++ b/go/fory/doc.go @@ -0,0 +1,336 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +/* +Package fory provides high-performance, cross-language serialization for Go. + +Fory is a blazingly fast multi-language serialization framework that enables +seamless data exchange between Go, Java, Python, C++, Rust, and JavaScript. +It supports automatic object graph serialization with circular references, +polymorphism, and schema evolution. + +# Requirements + +Go 1.24 or later is required. + +# Installation + + go get github.com/apache/fory/go/fory + +# Quick Start + +Create a Fory instance, register your types, and serialize: + + package main + + import ( + "fmt" + "github.com/apache/fory/go/fory" + ) + + type User struct { + ID int64 + Name string + Age int32 + } + + func main() { + // Create a Fory instance + f := fory.New() + + // Register struct with a type ID + if err := f.RegisterStruct(User{}, 1); err != nil { + panic(err) + } + + // Serialize + user := &User{ID: 1, Name: "Alice", Age: 30} + data, err := f.Serialize(user) + if err != nil { + panic(err) + } + + // Deserialize + var result User + if err := f.Deserialize(data, &result); err != nil { + panic(err) + } + + fmt.Printf("Deserialized: %+v\n", result) + } + +# Configuration Options + +Fory uses a functional options pattern for configuration: + + // Default configuration + f := fory.New() + + // With options + f := fory.New( + fory.WithTrackRef(true), // Enable reference tracking for circular references + fory.WithCompatible(true), // Enable schema evolution + fory.WithMaxDepth(30), // Set maximum nesting depth + fory.WithXlang(true), // Enable cross-language mode + ) + +Configuration defaults: + + - TrackRef: false (reference tracking disabled) + - MaxDepth: 20 (maximum nesting depth) + - IsXlang: false (cross-language mode disabled) + - Compatible: false (schema evolution disabled) + +# Type Registration + +Types must be registered before serialization. Registration can be done by ID +(compact, faster) or by name (flexible): + + // Register by ID (recommended for performance) + f.RegisterStruct(User{}, 1) + f.RegisterStruct(Order{}, 2) + + // Register by name (more flexible) + f.RegisterNamedStruct(User{}, "example.User") + + // Register enum types + f.RegisterEnum(Status(0), 3) + +All struct types in the object graph must be registered, including nested types. + +# Supported Types + +Fory Go supports a wide range of types: + +Primitives: + - bool, int8, int16, int32, int64, int + - uint8, uint16, uint32, uint64 + - float32, float64 + - string + +Collections: + - Slices: []T for any serializable type T + - Maps: map[K]V for supported key/value types + - Sets: fory.Set[T] (backed by map[T]struct{}) + +Time types: + - time.Time (nanosecond precision) + - time.Duration + +Structs: + - Any struct with exported fields + - Nested structs (must be registered) + - Pointer fields with nil handling + +Binary data: + - []byte + +# Struct Tags + +Fory supports struct tags for field-level configuration: + + type Document struct { + ID int64 `fory:"id=0"` // Field ID for compact encoding + Title string `fory:"id=1"` // Field IDs must be unique + Author *User `fory:"id=2,ref"` // Enable reference tracking + Password string `fory:"-"` // Exclude from serialization + Count int64 `fory:"encoding=fixed"` // Fixed-length encoding + Data *Data `fory:"nullable"` // Allow nil values + } + +Available tags: + - id=N: Assign numeric field ID for compact encoding + - "-": Exclude field from serialization + - ref: Enable reference tracking for pointer/slice/map fields + - nullable: Write null flag for pointer fields + - encoding=varint|fixed|tagged: Control numeric encoding + +# Reference Tracking + +Enable reference tracking to handle circular references and shared objects: + + f := fory.New(fory.WithTrackRef(true)) + + type Node struct { + Value int32 + Next *Node `fory:"ref"` // Enable per-field tracking + } + + // Create circular list + n1 := &Node{Value: 1} + n2 := &Node{Value: 2} + n1.Next = n2 + n2.Next = n1 // Circular reference + + data, _ := f.Serialize(n1) // Handles circular reference correctly + +When reference tracking is enabled: + - Objects appearing multiple times are serialized once + - Circular references are handled correctly + - Per-field `fory:"ref"` tags take effect + +# Schema Evolution + +Enable compatible mode for forward/backward compatibility: + + f := fory.New(fory.WithCompatible(true)) + + // V1: original struct + type UserV1 struct { + ID int64 + Name string + } + + // V2: added Email field + type UserV2 struct { + ID int64 + Name string + Email string // New field - receives zero value from V1 data + } + +Compatible mode supports: + - Adding new fields (receive zero values from old data) + - Removing fields (skipped during deserialization) + - Reordering fields (matched by name) + +# Cross-Language Serialization + +Enable cross-language mode for interoperability with Java, Python, C++, Rust, +and JavaScript: + + f := fory.New(fory.WithXlang(true)) + f.RegisterStruct(User{}, 1) // Use same ID across all languages + + data, _ := f.Serialize(&User{ID: 1, Name: "Alice"}) + // 'data' can be deserialized by Java, Python, etc. + +For cross-language compatibility: + - Use consistent type IDs across all languages + - Fields are matched by snake_case name conversion + - See https://fory.apache.org/docs/specification/xlang_type_mapping for type mappings + +# Thread Safety + +The default Fory instance is NOT thread-safe. For concurrent use, use the +threadsafe package: + + import "github.com/apache/fory/go/fory/threadsafe" + + // Create thread-safe instance + f := threadsafe.New( + fory.WithTrackRef(true), + fory.WithCompatible(true), + ) + + // Safe for concurrent use + go func() { + data, _ := f.Serialize(value1) + }() + go func() { + data, _ := f.Serialize(value2) + }() + +The thread-safe wrapper: + - Uses sync.Pool for efficient instance reuse + - Automatically copies serialized data before returning + - Accepts the same configuration options as fory.New() + +# Buffer Management + +The default Fory instance reuses its internal buffer for zero-copy performance: + + f := fory.New() + data1, _ := f.Serialize(value1) + // WARNING: data1 becomes invalid after next Serialize call! + data2, _ := f.Serialize(value2) + +To keep the data, copy it: + + safeCopy := make([]byte, len(data1)) + copy(safeCopy, data1) + +The thread-safe wrapper automatically copies data, avoiding this concern. + +# Code Generation (Experimental) + +For performance-critical paths, Fory provides optional ahead-of-time code +generation that eliminates reflection overhead: + + //fory:generate + type User struct { + ID int64 + Name string + } + + //go:generate fory -pkg . + +Run code generation: + + go generate ./... + +Generated code integrates transparently - Fory automatically uses generated +serializers when available. + +# Error Handling + +Always check errors from serialization operations: + + data, err := f.Serialize(value) + if err != nil { + // Handle error + } + + err = f.Deserialize(data, &result) + if err != nil { + // Handle error + } + +Common error kinds: + - ErrKindBufferOutOfBound: Read/write beyond buffer bounds + - ErrKindTypeMismatch: Type ID mismatch during deserialization + - ErrKindUnknownType: Unknown type encountered + - ErrKindMaxDepthExceeded: Recursion depth limit exceeded + - ErrKindHashMismatch: Struct hash mismatch (schema changed) + +# Best Practices + +1. Reuse Fory instances: Creating a Fory instance involves initialization +overhead. Create once and reuse. + +2. Use thread-safe wrapper for concurrency: Never share a non-thread-safe +Fory instance across goroutines. + +3. Enable reference tracking only when needed: It adds overhead for tracking +object identity. + +4. Copy serialized data if keeping it: With the default Fory, the returned +byte slice is invalidated on the next operation. + +5. Register all types at startup: Before any concurrent operations. + +6. Use compatible mode for evolving schemas: Enable when struct definitions +may change between service versions. + +# More Information + +For comprehensive documentation, see https://fory.apache.org/docs/guide/go/ + +Related specifications: + - Xlang Serialization: https://fory.apache.org/docs/specification/fory_xlang_serialization_spec + - Type Mapping: https://fory.apache.org/docs/specification/xlang_type_mapping +*/ +package fory