Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving a library file in 10.8 destroys all consolidated clips in the library. #376

Open
sefanzed opened this issue Jul 1, 2024 · 17 comments
Assignees

Comments

@sefanzed
Copy link

sefanzed commented Jul 1, 2024

Apple Feedback Assistant ID: MISSING

DESCRIBE THE BUG:
If you move your FCP library file and then reopen it, all of the consolidated clips in the project timeline are replaced with slugs. When you ask FCP to locate the clip in the browser, it reports that it can't locate the clip. Indeed, all of the clips in the library file are gone!


TO REPRODUCE:
Simply move your library file to a different location on your HD


EXPECTED BEHAVIOUR:
One would expect perhaps re-linking some media, if that. The entire destruction of every consolidated clip in the library is certainly not!!


SPECS:

  • 2022 Mac Studio 8TB HD 128 GB ram
  • macOS Sonoma 14.5
  • Final Cut Pro 10.8

ADDITIONAL COMMENTS:
Attempt to repro at your own peril!

@sefanzed
Copy link
Author

sefanzed commented Jul 2, 2024

I should also mention that the library exhibiting this behavior was two levels down in the finder.

@joema4
Copy link

joema4 commented Jul 9, 2024

I just tried this and it didn't happen. I tried two different consolidation methods, both using an edited timeline: (1) Imported some clips with "leave files in place" and some imported with "copy to library", and a media storage location external to the library designated by the Library Inspector>Storage Locations>Modify Settings>Media, with files consolidated to there, and (2) All clips imported with "leave files in place", consolidated to within the library.

In both cases, I shut down FCP, moved the library to another drive, started FCP and the media, and all the media was linked OK. Are all media and library files in this scenario on locally attached APFS or HFS+ volumes?

@sefanzed
Copy link
Author

sefanzed commented Jul 9, 2024

All of my media was NOT consolidated into the library. The behavior I was referring to affected only the consolidated group clips (Using option G) that combined both graphics and footage. It was certainly reproducable on my system (Sonoma 14.5-M1 Ultra Studio, as the same thing happened with two different libraries on my main drive (8TB SSD). After I dragged the libraries out of the folders where they originally existed (one was down a level, the other was down two levels) and reopened the re-located libraries, the media was all there; however, all of the consolidated group clips in the library were flagged as missing and indeed were entirely gone. In their place in the timeline were grey slugs. When I selected the slugs in the timeline and did a Shift-F to reveal the location of the group clips in the library browser, I got an error message saying that FCP couldn't locate them. I had to rebuild them. A real PITA.

@joema4
Copy link

joema4 commented Jul 9, 2024

Do you mean compound clips, not consolidated? If so, I'll start over and re-test those.

@sefanzed
Copy link
Author

sefanzed commented Jul 9, 2024

Yes. Sorry for the confusion.

@joema4
Copy link

joema4 commented Jul 10, 2024

I tried it using compound clips composed of both video and graphics. I moved the library to another disk volume, but the problem didn't happen. To save me some time, could you please answer these questions:

  • Were the compound clips originally created in that library or were they copied or dragged from another FCP library?
  • Please pick a representative compound clip, export a clip XML and post that .fcpxmld file here.
  • Was the library moved to another disk volume or just within the 8TB system volume?
  • Is the media imported using "leave files in place" or is it copied within the library?
  • Is the media on the 8TB system volume or another disk volume?

@sefanzed
Copy link
Author

sefanzed commented Jul 10, 2024 via email

@sefanzed
Copy link
Author

The compound clips were created from stacked clips in the timeline. The clips were in the library and resided on the same drive, in nested folders.
Please pick a representative compound clip, export a clip XML and post that .fcpxmld file here.
Attached.
Was the library moved to another disk volume or just within the 8TB system volume?
The library was dragged out of its original folder and up one (or two) level(s).
Is the media imported using "leave files in place" or is it copied within the library?
All media was imported with “leave files in place.”
Is the media on the 8TB system volume or another disk volume?
The media resided on the 8TB system volume. The hierarchy is: Desktop/”Final Cut Pro Projects”/projectname/”showassets”/"episode one"/library.
Opening Credits.fcpxmld.zip

@joema4
Copy link

joema4 commented Jul 10, 2024

Thanks, I studied the XML closely. That compound clip is relatively complex, containing two nested levels with other compound clips, basic titles, basic color Fx, noise reduction and retiming. However, there's nothing wrong with that. It only references two media clips, PR_101.mov and "Circle of Electricity Green Screen.mp4".

Just moving the FCP library should never make anything like this happen (ie compound clip contents vanishing and replaced with gray gap clips). Even if you move it to another hard drive, it should not happen. Unlike media links, compound clips are defined within the library. There is a separate SQLite database that stores the parent compound clip definition and a SQLite database for the project that references those. All that is within the library, and should be independent of library location.

I only see one possible reference to something like this, reported 5-24-24 on Creative Cow: https://creativecow.net/forums/thread/missing-empty-compound-clips-in-timeline/

Unfortunately, the cause was never isolated, and the user did not report any specific error messages. He didn't move the library, just restarted the machine and it happened.

Users commonly create gigantic numbers of deeply-nested compound clips and move libraries all the time. If there was any common failure mode it would have been reported long ago. I'm not saying you didn't experience a problem, and the Creative Cow post seems to indicate there might be some rare issue.

If you ever tried to move the library when FCP was up, in theory it could cause a library database problem. Unlike Windows, MacOS does not have "mandatory" file locking and any app can move a file anytime. If that was moved when FCP was still doing I/O it could cause a database inconsistency.

Unfortunately, there are no tools for checking library consistency, but there is an undocumented tool for checking the project. That's done by selecting the project in the left sidebar, holding down the OPT key and doing Clip>Verify and Repair Project. You could try this and see if it reports anything. Make sure everything is backed up before trying it, including a file-level backup of the library itself. If it reports an error, make a screen cap of that.

You said "When I selected the slugs in the timeline and did a Shift-F to reveal the location of the group clips in the library browser, I got an error message saying that FCP couldn't locate them" What was that exact message?

@sefanzed
Copy link
Author

I never saw this in 10.7, and I routinely moved libraries in and out of folders on the same drive, so this was a shocker—especially since I was still waiting on client comments... I also saw it previously with another project after I updated FCP to 10.8, but I wrote it off as "one of those things." However, seeing it again after performing the same operation confirmed (at least in my mind) that it was a bug in the update. Going forward, if I need to move the library, I'll create a new one and copy the events and projects into the new library location. And, to answer your last question, I believe it was "Unable to locate file."

@sefanzed
Copy link
Author

sefanzed commented Jul 10, 2024

I was reading the comments on the missing/empty compound clips on Creative Cow. That is quite different; in that case, the compound clips, while replaced with gap slugs in the project timeline, still existed in his browser. In my situation, the compound clips were entirely gone from both the browser and the timeline. It is also worth mentioning that FCP was not active when I moved the libraries and discovered the subsequent issue.

@joema4
Copy link

joema4 commented Jul 10, 2024

I understand. The compound clips in the browser are totally missing -- not replaced with gap or slug clips, the timeline version of those compound clips are replaced with gap/slug clips, and when you do SHIFT+F you get an error. What is that exact error?

This is even more confusing, as you cannot intentionally or by accident delete a browser compound clip which is used by any timeline. FCP knows there is a timeline using the clip and will raise the error: "Clips in Use, Final Cut Pro cannot delete multicam or compound clips that are in use."

At this point I need to see a copy of your library. It need not have any media -- I need to examine the internal database structures. I helped develop SQL Server at Microsoft so I am familiar with that. Please delete all cache files, proxy files and optimized media files using the FCP commands, shut down FCP and use the Finder "compress" feature to create a .zip version of that library. Then send it to me using this secure write-only link: https://www.dropbox.com/scl/fo/4qinmqamtvsq6ofh0ifwp/AEALDnKc-p70dudg0HtV3Io?rlkey=ret240t3lfvhuk7sdawr5bqaj&st=mouqxhzs&dl=0

@sefanzed
Copy link
Author

Let me know if you got it.

@joema4
Copy link

joema4 commented Jul 10, 2024

I got it, but I need to see a version in the damaged state. Do you have one or can you duplicate in Finder a non-damaged library, open that library and reproduce the problem, then zip and send me the damaged library version?

@sefanzed
Copy link
Author

Now, I can't get the problem to reoccur! I duplicated the library and dragged it all over the place. However, when I open it, everything is as expected! Not sure where to go with this... Sorry for making this such a wheel spinner!

@joema4
Copy link

joema4 commented Jul 10, 2024

Steve, no problem. We appreciate your report. The issue seemed serious enough to justify the effort. Since you have a well-equipped machine with no external storage, we can rule out problems in that area. Apart from the unlikely scenario of moving a library while Final Cut Pro is open, it should be impossible to cause this error through user actions. I don't recall encountering this in other user cases, and there are no apparent reports of it on Google.

If you have a backed-up library via Time Machine or an FCP auto-backup (by default in /Movies/Final Cut Backups), I could examine those. However, it would need to be a version with the problem. If you know the date and approx. time of day when it happened, there might be an auto-backup of the library from then.

The procedure would be to pick the auto-backup with a date/time close to what you remember, then double-click to open it in FCP and see if the problem exists on that version. This should be safe since it creates a new library using the backup name (where the name contains date/time and time zone). If you see the damaged version, send me that.

FCP data integrity is generally excellent (IMO far better than Premiere and Resolve), but it's not perfect. Even if it can't be reproduced, if you could find a library with that problem, I could inspect and document it. Use your own judgment about whether it's worth the time to pursue this.

@sefanzed
Copy link
Author

I went through my backups and couldn't locate any with the issue. Evidently, the backups were made AFTER I rebuilt the compound clips...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants