Skip to content

Conversation

HassanBahati
Copy link
Member

@HassanBahati HassanBahati commented Aug 26, 2025

Fixes #2506
This PR fixes the exponential growth of items for the backfill listing and excluded paths parameter not being applied during listing

@cabljac cabljac requested a review from Copilot August 26, 2025 14:21
@cabljac cabljac marked this pull request as ready for review August 26, 2025 14:22
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes issues in the storage-resize-images extension where the backfill process was experiencing exponential growth of items and excluded paths weren't being properly filtered during file listing operations.

  • Adds path exclusion filtering to prevent scanning of already-resized images during backfill
  • Implements optimized query handling with delimiter parameter for better performance
  • Introduces comprehensive test coverage for the backfill fix functionality

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
storage-resize-images/functions/src/index.ts Adds path exclusion functions and integrates them into backfill process
storage-resize-images/functions/tests/backfill-fix.test.ts New test file validating path exclusion logic and preventing exponential growth

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +157 to +167
function buildOptimizedQuery(baseQuery: any, excludePathList?: string[]): any {
if (!excludePathList || excludePathList.length === 0) {
return baseQuery;
}

const optimizedQuery = { ...baseQuery };
optimizedQuery.delimiter = "/";

return optimizedQuery;
}

Copy link
Preview

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buildOptimizedQuery function is defined but never used in the codebase. Consider removing it or integrating it into the getFilesWithPathExclusions function to avoid code duplication.

Suggested change
function buildOptimizedQuery(baseQuery: any, excludePathList?: string[]): any {
if (!excludePathList || excludePathList.length === 0) {
return baseQuery;
}
const optimizedQuery = { ...baseQuery };
optimizedQuery.delimiter = "/";
return optimizedQuery;
}

Copilot uses AI. Check for mistakes.

@@ -18,6 +18,7 @@ import * as admin from "firebase-admin";
import { getFunctions } from "firebase-admin/functions";
import { getExtensions } from "firebase-admin/extensions";
import * as functions from "firebase-functions/v1";
import { logger } from "firebase-functions";
Copy link
Preview

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger import is added but not used anywhere in the code. Remove this unused import.

Suggested change
import { logger } from "firebase-functions";

Copilot uses AI. Check for mistakes.

}

return files.filter((file) => {
const filePath = path.dirname(file.metadata.name);
Copy link
Preview

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using path.dirname() on a storage object name may not work correctly for files in the root directory, as it would return '.' instead of an empty string. Consider using file.metadata.name.split('/').slice(0, -1).join('/') or handle the root case explicitly.

Suggested change
const filePath = path.dirname(file.metadata.name);
const filePath = file.metadata.name.split('/').slice(0, -1).join('/');

Copilot uses AI. Check for mistakes.

for (const excludePath of excludePathList) {
const trimmedExcludePath = excludePath
.trim()
.replace(/\*/g, "([a-zA-Z0-9_\\-\\+.\\s\\/]*)?");
Copy link
Preview

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern allows arbitrary characters including forward slashes, which could lead to unintended path matching. Consider using a more restrictive character set or escaping special regex characters properly to prevent regex injection.

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐛 [storage-resize-images] executions & exponential costs in backfill mo
1 participant