Merge/main to develop #4484
Merge/main to develop #4484
Conversation
Add CentOS support to `fiftyone-db`
* add an effect to refetch saved views on dataset change * rm saved view attr usage for state, handle group slice --------- Co-authored-by: Benjamin Kane <[email protected]>
Updating release notes
Merge `release/v0.24.1` to `main`
WalkthroughRecent updates include adding a step to upload wheel artifacts conditionally in the GitHub workflow, introducing a Changes
Poem
Warning Review ran into problemsProblems (1)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Outside diff range and nitpick comments (5)
fiftyone/server/mutation.py (4)
Line range hint
210-210: Consider adding error handling for_runfunction call.The
_runfunction is called without any error handling. It's recommended to wrap this call in a try-except block to handle potential exceptions, especially since this function deals with file operations which are prone to errors.
Line range hint
796-796: Add error handling for_runfunction inmove_files.The
_runfunction is used here without error handling. Since this function is performing file move operations, which can fail due to permissions issues or file system errors, it's crucial to add error handling to manage these potential issues effectively.
Line range hint
851-851: Consider implementing error handling for_runindelete_files.This is another instance where
_runis used for batch file operations without error handling. Implementing error handling here can help in managing exceptions that might occur during the file deletion process, such as permissions errors or trying to delete non-existent files.
Line range hint
864-885: Refactor to improve error handling and readability.The
runfunction could benefit from improved error handling and readability. Consider refactoring to separate the concerns of progress tracking and the actual function execution. This would make the function easier to maintain and extend in the future.def run(fcn, tasks, return_results=True, num_workers=None, progress=None): try: num_tasks = len(tasks) except: num_tasks = None if num_tasks == 0: return [] if return_results else None num_workers = fou.recommend_thread_pool_workers(num_workers) kwargs = dict(total=num_tasks, iters_str="files", progress=progress) results = [] if num_workers <= 1: with fou.ProgressBar(**kwargs) as pb: for task in pb(tasks): try: result = fcn(task) if return_results: results.append(result) except Exception as e: handle_error(e) else: with multiprocessing.dummy.Pool(processes=num_workers) as pool: with fou.ProgressBar(**kwargs) as pb: for task in pb(pool.imap_unordered(fcn, tasks)): try: if return_results: results.append(task) except Exception as e: handle_error(e) return results if return_results else Noneapp/packages/operators/src/operators.ts (1)
Line range hint
55-55: Consider replacinganywith more specific types to enhance type safety and code maintainability.- json.requests.map((r: any) => InvocationRequest.fromJSON(r)), + json.requests.map((r: RawInvocationRequest) => InvocationRequest.fromJSON(r)), - const operatorInstances = operators.map((d: any) => + const operatorInstances = operators.map((d: RawInvocationRequest) => - const labels = []; + const labels: LabelType[] = []; - const items = []; + const items: AbortableOperation[] = []; - constructor(private items = []) {} + constructor(private items: AbortableOperation[] = []) {} - this.items.push(new AbortableOperation(uri, params, parser)); + this.items.push(new AbortableOperation(uri, params as ParamsType, parser)); - const items = this.findByURI(uri); + const items: AbortableOperation[] = this.findByURI(uri);Also applies to: 74-74, 109-109, 341-341, 434-434, 689-689, 905-905
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (13)
- .github/workflows/build-db.yml (1 hunks)
- app/packages/core/src/components/MainSpace/MainSpace.tsx (1 hunks)
- app/packages/core/src/components/Sidebar/ViewSelection/index.tsx (1 hunks)
- app/packages/operators/src/OperatorInvocationRequestExecutor.tsx (1 hunks)
- app/packages/operators/src/constants.ts (1 hunks)
- app/packages/operators/src/operators.ts (2 hunks)
- app/packages/spaces/src/hooks.ts (3 hunks)
- app/packages/spaces/src/state.ts (1 hunks)
- docs/source/release-notes.rst (1 hunks)
- fiftyone/core/storage.py (6 hunks)
- fiftyone/server/events/initialize.py (3 hunks)
- fiftyone/server/mutation.py (4 hunks)
- package/db/setup.py (4 hunks)
Files skipped from review due to trivial changes (2)
- .github/workflows/build-db.yml
- docs/source/release-notes.rst
Additional context used
Path-based instructions (7)
app/packages/operators/src/constants.ts (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/operators/src/OperatorInvocationRequestExecutor.tsx (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/core/src/components/MainSpace/MainSpace.tsx (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/spaces/src/state.ts (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/core/src/components/Sidebar/ViewSelection/index.tsx (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/spaces/src/hooks.ts (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.app/packages/operators/src/operators.ts (1)
Pattern
**/*.{ts,tsx}: Review the Typescript and React code for conformity with best practices in React, Recoil, Graphql, and Typescript. Highlight any deviations.
GitHub Check: lint / eslint
app/packages/operators/src/operators.ts
[warning] 55-55:
Unexpected any. Specify a different type
[warning] 74-74:
Unexpected any. Specify a different type
[warning] 109-109:
Unexpected any. Specify a different type
[warning] 341-341:
Unexpected any. Specify a different type
[warning] 434-434:
Unexpected any. Specify a different type
[warning] 689-689:
Unexpected any. Specify a different type
[warning] 905-905:
Unexpected any. Specify a different type
[warning] 905-905:
Unexpected any. Specify a different type
Additional comments not posted (14)
app/packages/operators/src/constants.ts (1)
10-15: The addition of theQueueItemStatusenum is well-structured and follows TypeScript best practices.#!/bin/bash # Description: Verify that the `QueueItemStatus` enum is used consistently across the application. # Test: Search for the enum usage. Expect: Consistent usage across the application. rg --type ts $'QueueItemStatus'app/packages/operators/src/OperatorInvocationRequestExecutor.tsx (2)
1-1: The new imports are correctly placed and necessary for the updated functionality.Also applies to: 6-6
10-14: The use ofuseMemoto filter pending requests is a good practice for performance optimization. Ensure that the dependency array correctly reflects all variables that determine the output.#!/bin/bash # Description: Verify that the dependency array for `useMemo` is correctly specified. # Test: Search for the `useMemo` usage in the file. Expect: Correct dependency specification. rg --type tsx $'useMemo' OperatorInvocationRequestExecutor.tsxapp/packages/core/src/components/MainSpace/MainSpace.tsx (2)
11-14: The updateduseSpaceshook now includesclearSpaces, which is a useful addition for managing spaces state.
20-20: Ensure that theuseEffecthook that depends onclearSpacesis correctly implemented and does not introduce unintended side effects.#!/bin/bash # Description: Verify the correct implementation of `useEffect` that uses `clearSpaces`. # Test: Search for the `useEffect` usage in the file. Expect: Correct implementation without side effects. rg --type tsx $'useEffect' MainSpace.tsxapp/packages/spaces/src/state.ts (1)
12-14: AllowingSpaceNodeJSONvalues to beundefinedinspacesAtomis a significant change. Ensure that all parts of the application that usespacesAtomhandleundefinedvalues correctly.#!/bin/bash # Description: Verify the correct handling of `undefined` values in `spacesAtom`. # Test: Search for the `spacesAtom` usage in the application. Expect: Correct handling of `undefined` values. rg --type ts $'spacesAtom'fiftyone/server/events/initialize.py (2)
143-143: Settingstate.group_slicetoNonein the exception block is a safe fallback. Good error handling practice.
135-135: Ensure thatstate.dataset.group_sliceis always defined before assignment.#!/bin/bash # Description: Verify that `group_slice` is always defined in the dataset object before being assigned. # Test: Search for dataset object definition and ensure `group_slice` is always initialized. ast-grep --lang python --pattern $'class Dataset { $$$ group_slice = $_ $$$ }'app/packages/core/src/components/Sidebar/ViewSelection/index.tsx (1)
81-83: The addition of theuseEffecthook to refetch data ondatasetNamechange is a good practice to ensure data consistency.app/packages/spaces/src/hooks.ts (2)
34-37: Proper initialization of state whenstateis undefined. This ensures that the default state is set correctly.
43-45: The addition ofclearSpacesfunction enhances modularity and allows for better state management by providing a method to reset the state.package/db/setup.py (2)
40-52: The addition of new MongoDB download links for CentOS ensures compatibility with more systems. Ensure that these URLs are valid and accessible.#!/bin/bash # Description: Verify the accessibility of the new MongoDB download URLs for CentOS. # Test: Attempt to fetch the headers of the URLs to ensure they are accessible. echo "Checking URLs for CentOS:" urls=( "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-5.0.4.tgz" "https://fastdl.mongodb.org/linux/mongodb-linux-aarch64-rhel82-5.0.22.tgz" "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel80-5.0.4.tgz" "https://fastdl.mongodb.org/linux/mongodb-linux-aarch64-rhel90-7.0.2.tgz" "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel90-7.0.2.tgz" ) for url in "${urls[@]}"; do curl -I "$url" | head -n 1 done
103-104: The update of Windows download links ensures compatibility with the latest MongoDB versions. Verify that these links are correct and lead to the intended files.#!/bin/bash # Description: Verify the correctness of the Windows download links. # Test: Check if the URLs point to the correct MongoDB versions. echo "Checking Windows download links:" urls=( "https://fastdl.mongodb.org/windows/mongodb-windows-x86_64-5.0.4.zip" ) for url in "${urls[@]}"; do curl -I "$url" | head -n 1 doneapp/packages/operators/src/operators.ts (1)
7-7: Import ofQueueItemStatusfrom "./constants" aligns with the refactoring mentioned in the summary.
| @@ -243,14 +242,7 @@ async def set_view( | |||
| result_view = _build_result_view(result_view, form) | |||
There was a problem hiding this comment.
Ensure proper error handling for _run function call.
Similar to the previous comment, the _run function here is used to read multiple files concurrently. Given the nature of file I/O operations, adding error handling could prevent the application from crashing and provide a better user experience by gracefully handling file read errors.
| @@ -207,7 +207,7 @@ def open_files(paths, mode="r", skip_failures=False, progress=None): | |||
| a list of open file-like objects | |||
| """ | |||
| tasks = [(p, mode, skip_failures) for p in paths] | |||
| return _run(_do_open_file, tasks, progress=progress) | |||
| return _run(_do_open_file, tasks, return_results=True, progress=progress) | |||
There was a problem hiding this comment.
Add error handling for _run function call.
The _run function is used here to open multiple files concurrently. Given the nature of file operations, it's crucial to add error handling to manage potential issues such as file access errors or non-existent files.
| @@ -239,7 +239,7 @@ | |||
| a list of file contents | |||
| """ | |||
| tasks = [(p, binary, skip_failures) for p in paths] | |||
| return _run(_do_read_file, tasks, progress=progress) | |||
| return _run(_do_read_file, tasks, return_results=True, progress=progress) | |||
There was a problem hiding this comment.
Implement error handling for _run function call.
This is another instance where _run is used for reading files. Adding error handling here can help in managing exceptions that might occur during the file reading process, such as permissions errors or corrupted files.
| @@ -793,7 +793,7 @@ | |||
| progress callback function to invoke instead | |||
| """ | |||
| tasks = [(i, o, skip_failures) for i, o in zip(inpaths, outpaths)] | |||
| _run(_do_move_file, tasks, progress=progress) | |||
| _run(_do_move_file, tasks, return_results=False, progress=progress) | |||
There was a problem hiding this comment.
Consider adding error handling for _run function in move_files.
The _run function is used here without error handling. Since this function is performing file move operations, which can fail due to permissions issues or file system errors, it's crucial to add error handling to manage these potential issues effectively.
| @@ -848,7 +848,7 @@ | |||
| progress callback function to invoke instead | |||
| """ | |||
| tasks = [(p, skip_failures) for p in paths] | |||
| _run(_do_delete_file, tasks, progress=progress) | |||
| _run(_do_delete_file, tasks, return_results=False, progress=progress) | |||
There was a problem hiding this comment.
Ensure proper error handling for _run function in delete_files.
This is another instance where _run is used for batch file operations without error handling. Implementing error handling here can help in managing exceptions that might occur during the file deletion process, such as permissions errors or trying to delete non-existent files.
| def run(fcn, tasks, return_results=True, num_workers=None, progress=None): | ||
| """Applies the given function to each element of the given tasks. | ||
|
|
||
| Args: | ||
| fcn: a function that accepts a single argument | ||
| tasks: an iterable of function arguments | ||
| return_results (True): whether to return the function results | ||
| num_workers (None): a suggested number of threads to use | ||
| progress (None): whether to render a progress bar (True/False), use the | ||
| default value ``fiftyone.config.show_progress_bars`` (None), or a | ||
| progress callback function to invoke instead | ||
|
|
||
| Returns: | ||
| the list of function outputs | ||
| """ | ||
| num_workers = fou.recommend_thread_pool_workers(num_workers) | ||
|
|
||
| try: | ||
| num_tasks = len(tasks) | ||
| except: | ||
| num_tasks = None | ||
|
|
||
| kwargs = dict(total=num_tasks, iters_str="files", progress=progress) | ||
|
|
||
| if num_workers <= 1: | ||
| with fou.ProgressBar(**kwargs) as pb: | ||
| results = [fcn(task) for task in pb(tasks)] | ||
| else: | ||
| with multiprocessing.dummy.Pool(processes=num_workers) as pool: | ||
| with fou.ProgressBar(**kwargs) as pb: | ||
| results = list(pb(pool.imap(fcn, tasks))) | ||
|
|
||
| return results | ||
| the list of function outputs, or None if ``return_results == False`` | ||
| """ | ||
| return _run( | ||
| fcn, | ||
| tasks, | ||
| return_results=return_results, | ||
| num_workers=num_workers, | ||
| progress=progress, | ||
| ) |
There was a problem hiding this comment.
Refactor to improve error handling and readability.
The run function could benefit from improved error handling and readability. Consider refactoring to separate the concerns of progress tracking and the actual function execution. This would make the function easier to maintain and extend in the future.
def run(fcn, tasks, return_results=True, num_workers=None, progress=None):
try:
num_tasks = len(tasks)
except:
num_tasks = None
if num_tasks == 0:
return [] if return_results else None
num_workers = fou.recommend_thread_pool_workers(num_workers)
kwargs = dict(total=num_tasks, iters_str="files", progress=progress)
results = []
if num_workers <= 1:
with fou.ProgressBar(**kwargs) as pb:
for task in pb(tasks):
try:
result = fcn(task)
if return_results:
results.append(result)
except Exception as e:
handle_error(e)
else:
with multiprocessing.dummy.Pool(processes=num_workers) as pool:
with fou.ProgressBar(**kwargs) as pb:
for task in pb(pool.imap_unordered(fcn, tasks)):
try:
if return_results:
results.append(task)
except Exception as e:
handle_error(e)
return results if return_results else NoneCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def run(fcn, tasks, return_results=True, num_workers=None, progress=None): | |
| """Applies the given function to each element of the given tasks. | |
| Args: | |
| fcn: a function that accepts a single argument | |
| tasks: an iterable of function arguments | |
| return_results (True): whether to return the function results | |
| num_workers (None): a suggested number of threads to use | |
| progress (None): whether to render a progress bar (True/False), use the | |
| default value ``fiftyone.config.show_progress_bars`` (None), or a | |
| progress callback function to invoke instead | |
| Returns: | |
| the list of function outputs | |
| """ | |
| num_workers = fou.recommend_thread_pool_workers(num_workers) | |
| try: | |
| num_tasks = len(tasks) | |
| except: | |
| num_tasks = None | |
| kwargs = dict(total=num_tasks, iters_str="files", progress=progress) | |
| if num_workers <= 1: | |
| with fou.ProgressBar(**kwargs) as pb: | |
| results = [fcn(task) for task in pb(tasks)] | |
| else: | |
| with multiprocessing.dummy.Pool(processes=num_workers) as pool: | |
| with fou.ProgressBar(**kwargs) as pb: | |
| results = list(pb(pool.imap(fcn, tasks))) | |
| return results | |
| the list of function outputs, or None if ``return_results == False`` | |
| """ | |
| return _run( | |
| fcn, | |
| tasks, | |
| return_results=return_results, | |
| num_workers=num_workers, | |
| progress=progress, | |
| ) | |
| def run(fcn, tasks, return_results=True, num_workers=None, progress=None): | |
| try: | |
| num_tasks = len(tasks) | |
| except: | |
| num_tasks = None | |
| if num_tasks == 0: | |
| return [] if return_results else None | |
| num_workers = fou.recommend_thread_pool_workers(num_workers) | |
| kwargs = dict(total=num_tasks, iters_str="files", progress=progress) | |
| results = [] | |
| if num_workers <= 1: | |
| with fou.ProgressBar(**kwargs) as pb: | |
| for task in pb(tasks): | |
| try: | |
| result = fcn(task) | |
| if return_results: | |
| results.append(result) | |
| except Exception as e: | |
| handle_error(e) | |
| else: | |
| with multiprocessing.dummy.Pool(processes=num_workers) as pool: | |
| with fou.ProgressBar(**kwargs) as pb: | |
| for task in pb(pool.imap_unordered(fcn, tasks)): | |
| try: | |
| if return_results: | |
| results.append(task) | |
| except Exception as e: | |
| handle_error(e) | |
| return results if return_results else None |
What changes are proposed in this pull request?
Merge
maintodevelopafter release v0.24.1Summary by CodeRabbit
New Features
Bug Fixes
Chores
Refactor
QueueItemStatusenum to a separate file for better organization.