Showing content from https://github.com/privacysandbox/fledge-key-value-service below:
privacysandbox/protected-auction-key-value-service: Protected Auction Key/Value Service
Important
The main branch hosts live code with latest changes. It is unstable and is used for development. It is suitable for contribution and inspection of the latest code. The release-* branches are stable releases that can be used to build and deploy the system.
FLEDGE has been renamed to Protected Audience API. To learn more about the name change, see the blog post.
Protected Auction Key/Value service
The current codebase represents the implementation of the TEE-based Key/Value service by Privacy Sandbox.
For Protected Audience, the service can be used as a BYOS KV server. Soon it can be used to communicate with Chrome and the Bidding and Auction services using V2 protocol.
For Protected App Signals, the service should be used as the ad retrieval server.
It can be integrated with Chrome and Android with the Privacy Sandbox unified origin trial and Privacy Sandbox on Android Developer Preview.
- Source code is available on Github
- Releases are done on a regular basis
- Binaries can be built from source code
- C++ binary
- [AWS & GCP] Docker container image
- [AWS]: Nitro EIF
- [AWS]: Reference AMI
- Other tools
- Server can run as a standalone local process for testing without any cloud dependency or TEE-related functionality
- Server can be deployed to AWS Nitro enclave
- Server can be deployed to GCP Confidential Space
- Reference terraform available for a clean and comprehensive deployment to AWS or GCP
- Clean: assumes the cloud environment has no preset infrastructure
- Comprehensive: deploys all dependencies and some recommended (but not necessarily required) configuration
- Many server behaviors can be configured by parameter flags without having to rebuild
- Server loads key/value data from cloud file storage
- Server loads key/value data from cloud pub/sub services
- Server loads data into an in-RAM representation for query serving
- Server continuously monitors for new data and incrementally updates ("delta files") the in-RAM representation
- Support independent data ingestion pipelining by monitoring directories in cloud file storage independently
- Supports Flatbuffers as the data event format
- Supports Avro and Riegeli as the data file format
- Supports snapshot files for faster server start up
- Users can perform compactions of delta files into snapshot files in an offline path
- Support Protected Audience Key Value Server query spec: can be used as a BYOS server to serve requests from Chrome
- Support simple key value lookups for queries
- Users can write "user defined functions" to execute custom logic to process queries
- User defined functions can be written in JavaScript or WASM
- User defined functions can call "GetValues" to look up key value from the dataset
- Set-as-a-value is supported
- A key "value" pair in the dataset can be a key and a set of values
- UDF can call "RunQuery" API to run set operations on sets (intersection, union, difference)
- For GCP, Terraform supports deploying into an existing VPC, such as used by the Bidding and Auction services Non-prod Server logs are persisted after server shutdown
- Data can be sharded and different servers may load and serve different shards (subset) of the dataset.
- Sharding supports data locality, where the operator specifies "sharding key" for key value pairs so different key value pairs can have the same sharding key.
The following sections include the timelines for the Trusted Key Value Server for Protected Auctions. Protected Auctions refer to Protected Audiences and Protected App Signals ad targeting products.
Beta testing General availability Enforcement For Protected Audience
(web browser on desktop)
July 2024
The Privacy-Sandbox-provided Key Value Server implementation can
- run as a BYOS KV server
- support production scale traffic and common functionalities
Q4 2024
Opt-in TEE mode will be available to the Adtechs. Opt-in guidance will be published in early Q4 2024.
No sooner than Q3 2025 Beta testing General availability For Protected Audience
(With Bidding & Auction services for Chrome or Android)
July 2024
The Privacy-Sandbox-provided Key Value Server implementation can be used with the Bidding and Auction services and
- run as a BYOS KV server
- support production scale traffic and common functionalities
Dec 2024
The Privacy-Sandbox-provided Key Value Server implementation can be used with the Bidding and Auction services and adtechs can opt-in TEE mode
For Protected App Signals June 2024
The Privacy-Sandbox-provided Key Value Server implementation supports Ad retrieval server functionality and protected communication for live traffic testing
Dec 2024
The implementation supports live traffic at scale
- For AWS, Terraform supports deploying into an existing VPC, such as the one that is used by the Bidding and Auction services
- Internal load balancer is used for servers to send metrics to OpenTelemetry collector
- In v0.16, the communication goes through a public load balancer
Integration with the Bidding & Auction services
- The Bidding and Auction services can send encrypted requests to the Key Value Server for Protected App Signals
- Consented Debugging is supported
- Diagnose tool to check the cloud environment to warn for potential setup errors before the system is deployed
- Operational playbook
- Introduction of unsafe metrics
- Unsafe metrics have privacy protections such as differential privacy noises
- More metrics for comprehensive SLO monitoring
- Data loading error handling
- The system can be configured to use different error handling strategy for different dataset
- Benchmarking tools
- Cost explainer
- Sets-as-values will switch to using bitsets to represent sets for faster RunQuery performance.
- Commitment to support window for active releases
- Update to V2 protocol to support the hybrid mode of BYOS & Opt-in TEE
- Chrome and Key Value server can communicate in the updated V2 protocol
- Chrome can send full publisher URL to TEE KV server under V2 protocol
H2 2024 Android-PA GA, PAS GA
- UDF can perform Key/Value lookup asynchronously
- Flags can be passed from the server parameters into UDF
- One Key Value Server system can be used for multiple use cases. Multiple UDFs can be loaded. Different UDF can be selected based on the request type.
- Canaring support for UDF: canary version UDF can be staged in machines with specific tags.
- First class support for customization of the system (without violating the trust model)
- Diagnose tool to collect standard and necessary debug information for troubleshooting requests
- Complete end to end example as a template to set up the service
Backward-incompatible changes are expected to be rare and will result in a major version change.
Contributions are welcome, and we will publish more detailed guidelines soon. In the meantime, if you are interested, open a new Issue in the GitHub repository.
The FLEDGE K/V feature set, API and system design are under active discussion and subject to change in the future. If you try this API and have feedback, we'd love to hear it:
General announcements will be made on the FLEDGE mailing list on chromium.org and the Privacy Sandbox progress update page on the Android developer site, if needed, an accompanying article will be published on the Chrome Developers blog and Android Developers blog.
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.3