Enable conformance testing for GCP blob store #79

Yaminik1996 · 2025-10-15T18:44:41Z

No description provided.

codecov-commenter · 2025-10-15T18:52:59Z

Codecov Report

❌ Patch coverage is 91.89189% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.21%. Comparing base (534ea0b) to head (86b7c51).

Files with missing lines	Patch %	Lines
.../salesforce/multicloudj/blob/gcp/GcpBlobStore.java	90.62%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main      #79      +/-   ##
============================================
+ Coverage     78.10%   78.21%   +0.11%     
- Complexity     1690     1701      +11     
============================================
  Files           136      136              
  Lines          6864     6890      +26     
  Branches        779      783       +4     
============================================
+ Hits           5361     5389      +28     
+ Misses         1079     1078       -1     
+ Partials        424      423       -1

Flag	Coverage Δ
unittests	`78.21% <91.89%> (+0.11%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sandeepvinayak · 2025-10-15T20:44:27Z

blob/blob-client/src/test/java/com/salesforce/multicloudj/blob/client/AbstractBlobStoreIT.java

    public void initializeWireMockServer() {
        harness = createHarness();
-        TestsUtil.startWireMockServer("src/test/resources", harness.getPort());
+    String path = String.format("src/test/resources", harness.getProviderId());


shouldn't break the pattern, you can revert this

Good catch, will revert this

sandeepvinayak · 2025-10-15T20:45:27Z

blob/blob-client/src/test/java/com/salesforce/multicloudj/blob/client/AbstractBlobStoreIT.java


        // And run the tests given the invalid credentialsOverrider
        runOperationsThatShouldFail("testInvalidCredentials", bucketClient);
+    if (harness.getProviderId() != GcpConstants.PROVIDER_ID) {


why do we need this ?

This PR was created at the time when presigned URL generation was failing for GCP blob store. I will enable this and other presigned URL tests in a separate PR.

sandeepvinayak · 2025-10-15T20:46:37Z

blob/blob-gcp/src/test/resources/delete-1wyfk7bmao.json

+{
+  "id" : "f6149961-1023-4bb1-902a-c3f650a5c5a2",
+  "name" : "storage_v1_b_substrate-sdk-gcp-poc1-test-bucket-versioned_o_conformance-tests_upload_happypath_versioned_path",
+  "request" : {
+    "url" : "/storage/v1/b/substrate-sdk-gcp-poc1-test-bucket-versioned/o/conformance-tests%2Fupload%2FhappyPath_versioned_Path",
+    "method" : "DELETE"
+  },
+  "response" : {
+    "status" : 204,
+    "headers" : {
+      "Alt-Svc" : "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
+      "Server" : "UploadServer",
+      "Cache-Control" : "no-cache, no-store, max-age=0, must-revalidate",
+      "X-GUploader-UploadID" : "AAwnv3J6BCE-ay8JyAKYXE7-xn8MKpGbTAIqiP3VEdYILA6EF9OXXg8ckmNEBe-tDN_lnsLJ",
+      "Vary" : [ "Origin", "X-Origin" ],
+      "Pragma" : "no-cache",
+      "Expires" : "Mon, 01 Jan 1990 00:00:00 GMT",
+      "Date" : "Fri, 03 Oct 2025 00:48:00 GMT",
+      "Content-Type" : "application/json"
+    }
+  },
+  "uuid" : "f6149961-1023-4bb1-902a-c3f650a5c5a2",
+  "persistent" : true,
+  "insertionIndex" : 1
+}


these should be in mappings, something is not right with the local configs while recording

They are some extra junk files, removed them. Please check mappings folder now.

sandeepvinayak · 2025-10-15T20:48:16Z

blob/blob-gcp/src/main/java/com/salesforce/multicloudj/blob/gcp/GcpBlobStore.java


    @Override
    protected void doDelete(String key, String versionId) {
+        validateBucketExists();


why do we need this validation separately? it's another call for any given operation. Isn't gcp throw resource not found otherwise ?

No, GCP does not throw an exception if bucket does not exist, rather the delete API just passes with a False return value. The integration test expects explicit exception to be thrown.

sandeepvinayak · 2025-10-15T20:52:09Z

blob/blob-gcp/src/main/java/com/salesforce/multicloudj/blob/gcp/GcpBlobStore.java


-        return new Iterator<>() {
-            private final Iterator<Blob> it = blobs.iterator();
+        Iterator<Blob> filteredIterator = applyDelimiterFilter(blobs, request.getPrefix(), request.getDelimiter());


why aren't we leveraging delimeter from list call ?
https://cloud.google.com/storage/docs/json_api/v1/objects/list

Taking a look at this

The behavior of GCS APIs with delimiter for filtering is not consistent. We are already making use of the delimiter and sending it as an API parameter. However, the expected filtering results are achieved only when the delimiter is slash(/), non-slash delimiters are not handled the same way as slash delimiter.

One example is: When the query is with delimiter="-" and prefix="conformance-tests/blob-for-list/prefix", list of keys of the available items is ["conformance-tests/blob-for-list", "conformance-tests/blob-for-list/prefix-1", "conformance-tests/blob-for-list/prefix-2", "conformance-tests/blob-for-list/prefix_3"], expected result is ["conformance-tests/blob-for-list/prefix_3"], but the GCS API call returns ["conformance-tests/blob-for-list/prefix-" "conformance-tests/blob-for-list/prefix_3"].

Another way to handle this is to relax/remove the integration test for this particular case.

sandeepvinayak · 2025-10-16T22:25:00Z

blob/blob-client/pom.xml

+        <dependency>
+            <groupId>com.salesforce.multicloudj</groupId>
+            <artifactId>multicloudj-common-gcp</artifactId>
+            <scope>test</scope>
+        </dependency>


this is not right dependency, client cannot depend on substrate specific dependencies

sandeepvinayak · 2025-10-16T22:27:18Z

blob/blob-client/src/test/java/com/salesforce/multicloudj/blob/client/AbstractBlobStoreIT.java

 import com.salesforce.multicloudj.blob.driver.UploadResponse;
 import com.salesforce.multicloudj.common.exceptions.InvalidArgumentException;
 import com.salesforce.multicloudj.common.exceptions.SubstrateSdkException;
+import com.salesforce.multicloudj.common.gcp.GcpConstants;


this is also not a right dependency, since this a temp workaround, just have a constant local and let's not introduce anti-pattern dependencies

salesforce-cla bot added the cla:signed label Oct 15, 2025

Yaminik1996 force-pushed the W-19364200/gcpBlobstoreConformanceTests branch 2 times, most recently from aa5a044 to d051128 Compare October 15, 2025 20:35

Yaminik1996 marked this pull request as ready for review October 15, 2025 20:43

sandeepvinayak reviewed Oct 15, 2025

View reviewed changes

Enable conformance tests on GCP blob store

dac269c

Yaminik1996 force-pushed the W-19364200/gcpBlobstoreConformanceTests branch from d051128 to dac269c Compare October 16, 2025 18:39

Enable conformance tests on GCP blob store

86b7c51

sandeepvinayak reviewed Oct 16, 2025

View reviewed changes

Enable conformance tests on GCP Blob Store

7b954da

Enable conformance testing for GCP blob store #79

Are you sure you want to change the base?

Enable conformance testing for GCP blob store #79

Uh oh!

Conversation

Yaminik1996 commented Oct 15, 2025

Uh oh!

codecov-commenter commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yaminik1996 Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sandeepvinayak Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Oct 15, 2025 •

edited

Loading

Yaminik1996 Oct 16, 2025 •

edited

Loading

sandeepvinayak Oct 16, 2025 •

edited

Loading