Fix/oci multi layer support#217
Merged
Merged
Conversation
Contributor
Author
|
@microsoft-github-policy-service agree company="Brainstorm Limitless, LLC" |
Contributor
|
Thanks for contributing. As discussed in the discord server, I will defer the PR review to @thomastaylor312 |
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR implements full OCI Distribution Specification v1.1 compliance by adding support for multi-layer OCI artifacts, enabling components to be distributed with security policies as atomic units and fixing critical bugs in policy attachment during dynamic loading.
- Added multi-layer OCI artifact support with SHA256 digest verification and timeout protection
- Fixed automatic policy attachment during runtime component loading (not just startup)
- Implemented CNCF WebAssembly OCI artifact specification compliance with wasip1/wasip2 support
Reviewed Changes
Copilot reviewed 9 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/wassette/src/oci_multi_layer.rs | Core multi-layer OCI support with digest verification and CNCF spec compliance |
| crates/wassette/src/loader.rs | Updated component loader to handle multi-layer artifacts and copy co-located policies |
| crates/wassette/src/lib.rs | Added timeout configuration and fixed policy auto-attachment during dynamic loading |
| tests/qr_generator_integration_test.rs | Integration tests using real multi-layer QR generator component |
| tests/oci_multi_layer_test.rs | OCI multi-layer functionality tests with registry health checks |
| tests/oci_digest_verification_test.rs | SHA256 digest verification tests for security compliance |
| tests/oci_cncf_spec_test.rs | CNCF WebAssembly OCI artifact specification compliance tests |
| crates/wassette/Cargo.toml | Added hex and sha2 dependencies for digest verification |
| Cargo.toml | Added workspace-level hex and sha2 dependencies |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Comment on lines
+174
to
+185
| let reference: oci_client::Reference = "registry.mcpsearchtool.com/test/qr-generator:v1755367253" | ||
| .parse()?; | ||
|
|
||
| let client = oci_client::Client::default(); |
There was a problem hiding this comment.
The test creates a new OCI client without timeout configuration, which could hang indefinitely. Consider using the same timeout configuration as in the main library (30 seconds) for consistency and to prevent test hangs.
Suggested change
| let reference: oci_client::Reference = "registry.mcpsearchtool.com/test/qr-generator:v1755367253" | |
| .parse()?; | |
| let client = oci_client::Client::default(); | |
| let client = oci_client::Client::with_timeout(Duration::from_secs(30)); |
Comment on lines
+131
to
+132
| if image_manifest.config.media_type == CONFIG_MEDIA_TYPE | ||
| || image_manifest.config.media_type == "application/vnd.oci.image.config.v1+json" { |
There was a problem hiding this comment.
The hardcoded media type string should be defined as a constant for consistency and maintainability, similar to how CONFIG_MEDIA_TYPE is defined.
Suggested change
| if image_manifest.config.media_type == CONFIG_MEDIA_TYPE | |
| || image_manifest.config.media_type == "application/vnd.oci.image.config.v1+json" { | |
| || image_manifest.config.media_type == OCI_IMAGE_CONFIG_MEDIA_TYPE { |
Comment on lines
+220
to
+264
| // Check if this is a multi-layer artifact issue | ||
| let error_str = e.to_string(); | ||
| if error_str.contains("Incompatible layer media type") { | ||
| // Multi-layer artifact detected - use our custom handler | ||
| info!("Multi-layer OCI artifact detected, using direct OCI client"); | ||
|
|
||
| // Use our new multi-layer support to get ALL layers | ||
| let artifact = crate::oci_multi_layer::pull_multi_layer_artifact(&reference, oci_client) | ||
| .await | ||
| .context("Failed to extract layers from multi-layer OCI artifact")?; | ||
|
|
||
| // Save the WASM data | ||
| let component_name = reference.repository().replace('/', "_"); | ||
| let (downloaded_resource, mut file) = DownloadedResource::new_temp_file( | ||
| &component_name, | ||
| Self::FILE_EXTENSION, | ||
| ) | ||
| .await?; | ||
|
|
||
| file.write_all(&artifact.wasm_data).await?; | ||
| file.flush().await?; | ||
| file.sync_all().await?; | ||
| drop(file); | ||
|
|
||
| // If there's a policy, save it alongside the WASM in the temp directory | ||
| if let Some(policy_data) = artifact.policy_data { | ||
| info!("Saving policy layer alongside component"); | ||
|
|
||
| // Create policy file in the same temp directory as the WASM | ||
| if let DownloadedResource::Temp((ref tempdir, ref _wasm_path)) = downloaded_resource { | ||
| let policy_path = tempdir.path().join(format!("{}.policy.yaml", component_name)); | ||
| tokio::fs::write(&policy_path, &policy_data) | ||
| .await | ||
| .context("Failed to save policy file")?; | ||
| info!("Policy saved to: {:?}", policy_path); | ||
| } | ||
| } | ||
|
|
||
| info!("Successfully extracted WASM component and policy from multi-layer artifact"); | ||
|
|
||
| Ok(downloaded_resource) |
There was a problem hiding this comment.
String-based error detection is fragile and could break if the error message format changes. Consider using structured error types or error codes for more reliable error handling.
Note: See the diff below for a potential fix:
@@ -217,47 +218,51 @@
Ok(downloaded_resource)
}
Err(e) => {
- // Check if this is a multi-layer artifact issue
- let error_str = e.to_string();
- if error_str.contains("Incompatible layer media type") {
- // Multi-layer artifact detected - use our custom handler
- info!("Multi-layer OCI artifact detected, using direct OCI client");
-
- // Use our new multi-layer support to get ALL layers
- let artifact = crate::oci_multi_layer::pull_multi_layer_artifact(&reference, oci_client)
- .await
- .context("Failed to extract layers from multi-layer OCI artifact")?;
-
- // Save the WASM data
- let component_name = reference.repository().replace('/', "_");
- let (downloaded_resource, mut file) = DownloadedResource::new_temp_file(
- &component_name,
- Self::FILE_EXTENSION,
- )
- .await?;
-
- file.write_all(&artifact.wasm_data).await?;
- file.flush().await?;
- file.sync_all().await?;
- drop(file);
-
- // If there's a policy, save it alongside the WASM in the temp directory
- if let Some(policy_data) = artifact.policy_data {
- info!("Saving policy layer alongside component");
+ // Check if this is a multi-layer artifact issue using structured error type
+ if let Some(oci_wasm_error) = e.downcast_ref::<OciWasmError>() {
+ if let OciWasmError::IncompatibleLayerMediaType = oci_wasm_error {
+ // Multi-layer artifact detected - use our custom handler
+ info!("Multi-layer OCI artifact detected, using direct OCI client");
- // Create policy file in the same temp directory as the WASM
- if let DownloadedResource::Temp((ref tempdir, ref _wasm_path)) = downloaded_resource {
- let policy_path = tempdir.path().join(format!("{}.policy.yaml", component_name));
- tokio::fs::write(&policy_path, &policy_data)
- .await
- .context("Failed to save policy file")?;
- info!("Policy saved to: {:?}", policy_path);
+ // Use our new multi-layer support to get ALL layers
+ let artifact = crate::oci_multi_layer::pull_multi_layer_artifact(&reference, oci_client)
+ .await
+ .context("Failed to extract layers from multi-layer OCI artifact")?;
+
+ // Save the WASM data
+ let component_name = reference.repository().replace('/', "_");
+ let (downloaded_resource, mut file) = DownloadedResource::new_temp_file(
+ &component_name,
+ Self::FILE_EXTENSION,
+ )
+ .await?;
+
+ file.write_all(&artifact.wasm_data).await?;
+ file.flush().await?;
+ file.sync_all().await?;
+ drop(file);
+
+ // If there's a policy, save it alongside the WASM in the temp directory
+ if let Some(policy_data) = artifact.policy_data {
+ info!("Saving policy layer alongside component");
+
+ // Create policy file in the same temp directory as the WASM
+ if let DownloadedResource::Temp((ref tempdir, ref _wasm_path)) = downloaded_resource {
+ let policy_path = tempdir.path().join(format!("{}.policy.yaml", component_name));
+ tokio::fs::write(&policy_path, &policy_data)
+ .await
+ .context("Failed to save policy file")?;
+ info!("Policy saved to: {:?}", policy_path);
+ }
}
+
+ info!("Successfully extracted WASM component and policy from multi-layer artifact");
+
+ Ok(downloaded_resource)
+ } else {
+ // Some other oci_wasm error - propagate it
+ Err(e)
}
-
- info!("Successfully extracted WASM component and policy from multi-layer artifact");
-
- Ok(downloaded_resource)
} else {
// Some other error - propagate it
Err(e)
f1890cf to
3090300
Compare
Contributor
Author
|
I'll work on getting the checks passing. it appears my dev setup does not match completely. |
3090300 to
a19b4ec
Compare
Contributor
|
@attackordie , hey let me know how I can help! I would love to have this PR merged |
Contributor
Author
|
Thanks for the offer. I have some time today for the project and might find you on discord with some questions. |
This commit introduces full support for multi-layer OCI images in wassette, expanding beyond single-layer WASM components. The implementation includes proper layer extraction, validation, and testing infrastructure while maintaining backwards compatibility with existing single-layer artifacts. Modified files: - crates/wassette/src/oci_multi_layer.rs - Enhanced multi-layer OCI handling - tests/oci_cncf_spec_test.rs - Updated CNCF specification compliance tests - tests/oci_integration_test.rs - Enhanced integration test scenarios - tests/oci_unit_test.rs - Expanded unit test coverage - Justfile - Added new test commands and GHCR testing support - Cargo.lock - Updated dependencies for improved OCI registry support New files: - Complete multi-layer OCI image processing capabilities - GHCR backwards compatibility testing with secure token handling - Extensive test suites covering multi-layer scenarios The multi-layer support enables more complex component distribution patterns while preserving the security model and performance characteristics of the existing single-layer implementation.
a19b4ec to
32bf6d7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix OCI multi-layer support for components with policies
Summary
This PR implements full OCI Distribution Specification v1.1 compliance by adding support for multi-layer OCI artifacts. Components can now be distributed with their security policies as a single atomic OCI artifact, fixing a critical bug where policies weren't automatically attached during dynamic component loading.
Fixes #216
Problems Fixed
This PR addresses multiple issues:
OCI Multi-layer Support - The OCI loader only extracted the first layer from OCI artifacts, ignoring additional layers like security policies (due to limitations in the
oci-wasmcrate)Dynamic Loading Bug - Policies from OCI artifacts weren't automatically attached when components were dynamically loaded at runtime, though they worked correctly at startup
Missing Digest Verification - No integrity checking was performed on downloaded OCI layers, creating a security vulnerability
Registry Timeout Issues - OCI operations could hang indefinitely on slow or unresponsive registries
CNCF WebAssembly OCI Spec Compliance - Didn't fully support the emerging WebAssembly OCI artifact specifications
Poor Error Messages - Users received cryptic "Incompatible layer media type" errors without understanding the root cause
Solution
New
oci_multi_layermodule that:oci-clientto fetch manifests and blobsFixed policy auto-attachment in
LifecycleManager::load_component:CNCF WebAssembly OCI compliance:
application/vnd.wasm.config.v0+jsonandapplication/vnd.wasm.config.v1+jsonChanges
New Files
crates/wassette/src/oci_multi_layer.rs- Core multi-layer OCI support implementationtests/oci_multi_layer_test.rs- Integration tests for multi-layer artifactstests/oci_cncf_spec_test.rs- CNCF WebAssembly OCI artifact spec compliance teststests/qr_generator_integration_test.rs- Real-world multi-layer component testsModified Files
crates/wassette/src/loader.rs- Updated to use new multi-layer loadercrates/wassette/src/lib.rs- Added module declaration and fixed policy auto-attachment bug.gitignore- Added patterns for test artifactsFeatures Implemented
✅ Full multi-layer OCI artifact support - Processes all layers based on media type
✅ Automatic policy extraction - Policy layers are automatically detected and attached
✅ SHA256 digest verification - Each layer's integrity is verified
✅ Backward compatibility - Single-layer WASM artifacts continue to work
✅ Timeout configuration - 30-second timeout prevents hanging on registry operations
Supported Media Types
application/wasm- WebAssembly componentsapplication/vnd.wassette.policy+yaml- Security policiesapplication/vnd.dev.cosign.simplesigning.v1+json- Cosign signaturesapplication/vnd.cncf.notary.signature- Notation signaturesapplication/vnd.in-toto+json- In-toto attestationsInstallation & Testing
Building from Source
Installing the Binary
Running Tests
Test Results (All Passing ✅)
test_multi_layer_with_policy_registry- Registry.mcpsearchtool.com supporttest_single_layer_wasm_compatibility- Backward compatibilitytest_load_component_with_policy_from_oci- Automatic policy attachmentSecurity Enhancements
Usage Example
Before this PR:
# Had to load component and policy separately wassette load-component oci://registry/component:tag wassette attach-policy file:///path/to/policy.yaml component-idAfter this PR:
Real-World Testing Example
With an AI Assistant (Claude, GitHub Copilot, etc.) using Wassette MCP:
The component and its security policy are loaded atomically from the OCI registry, demonstrating the new multi-layer support.
Impact and Benefits
Immediate Benefits
Enables Future Features
application/vnd.dev.cosign.simplesigning.v1+jsonapplication/vnd.cncf.notary.signatureBreaking Changes
None. This PR maintains full backward compatibility.
Migration Guide
No migration needed. Existing single-layer OCI artifacts continue to work exactly as before.
Checklist
Related Issues and PRs