Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backup: simple NAS backup plugin for KVM #9451

Merged
merged 75 commits into from
Sep 5, 2024
Merged

backup: simple NAS backup plugin for KVM #9451

merged 75 commits into from
Sep 5, 2024

Commits on Jul 26, 2024

  1. backup: Simple NAS backup plugin for KVM 

    This is an experimental simple NAS backup plugin for KVM which may be
later expanded for other hypervisors. This backup plugin aims to use
shared NAS storage on KVM hosts such as NFS or CephFS, which is used
to backup fully cloned VMs for backup & restore operations. This
may not be as efficient and performant as some of the other B&R
providers, but maybe useful for some KVM environments.

Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    f2baa68 View commit details
    Browse the repository at this point in the history

  2. cleanup and leave TODOs 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    2604f81 View commit details
    Browse the repository at this point in the history

  3. cleanup, refactor; add plugin config keys 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    b961a4d View commit details
    Browse the repository at this point in the history

  4. WIP B&R tool 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    9a8bbbb View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Merge remote-tracking branch 'origin/main' into kvm-backup-plugin-sha… 

    …red-storage
    @rohityadavcloud
    rohityadavcloud committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    aa90b6d View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. add cmd-answer classes 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    d5503e7 View commit details
    Browse the repository at this point in the history

  2. barebone take backup implementation 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f3bc039 View commit details
    Browse the repository at this point in the history

  3. prototype backup e2e 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f13769d View commit details
    Browse the repository at this point in the history

  4. fix backup script 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    1933f9e View commit details
    Browse the repository at this point in the history

  5. fix command execution handling 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    e066c5a View commit details
    Browse the repository at this point in the history

  6. implement backup delete 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    50e407c View commit details
    Browse the repository at this point in the history

  7. fix nasbackup script 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    9b0be25 View commit details
    Browse the repository at this point in the history

  8. fix op typo 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    11cbea2 View commit details
    Browse the repository at this point in the history

  9. add todos 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    18e0589 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. introduce backup repository concept for NAS & other backups 

    Each backup repository is for a specific zone & provider.
Eack NAS bkcp offering is tied to a back repo & each bkup offer assinged
to VM -> defined its backup repository

Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    97ce34c View commit details
    Browse the repository at this point in the history

  2. complete poc 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    0b9db75 View commit details
    Browse the repository at this point in the history

  3. fix backup script 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    00b22d4 View commit details
    Browse the repository at this point in the history

  4. ui tweaks

    @rohityadavcloud
    rohityadavcloud committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    7db5178 View commit details
    Browse the repository at this point in the history

  5. fix listbackups to return their original offering 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    452bc10 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Merge branch 'main' of https://github.com/apache/cloudstack into kvm-… 

    …backup-plugin-shared-storage
    @Pearl1594
    Pearl1594 committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    67821d7 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. add APIs for management of backup repositories and backing up from lo… 

    …cal stores and stopped VMs
    @Pearl1594
    Pearl1594 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    fe5c9d7 View commit details
    Browse the repository at this point in the history

  2. add license

    @Pearl1594
    Pearl1594 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    90b9981 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. support to backup stopped vms

    @Pearl1594
    Pearl1594 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    0cf5046 View commit details
    Browse the repository at this point in the history

  2. license

    @Pearl1594
    Pearl1594 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    427d89b View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. support restoring VMs

    @Pearl1594
    Pearl1594 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    e38baa7 View commit details
    Browse the repository at this point in the history

  2. add license

    @Pearl1594
    Pearl1594 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    90a45f9 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. support recover volume from backup and attach to VM

    @Pearl1594
    Pearl1594 committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    119f5ac View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. fix issues identified during dev testing

    @Pearl1594
    Pearl1594 committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    6483284 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Merge remote-tracking branch 'origin/main' into kvm-backup-plugin-sha… 

    …red-storage
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    ed9956f View commit details
    Browse the repository at this point in the history

  2. fix list API bugs 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    9889c44 View commit details
    Browse the repository at this point in the history

  3. remove unused import 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    123c911 View commit details
    Browse the repository at this point in the history

  4. api: restrict API to only root admin 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    0d217f5 View commit details
    Browse the repository at this point in the history

  5. ui: add ability to list & delete specific backup repository 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    5cb4f75 View commit details
    Browse the repository at this point in the history

  6. fixes 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    fc75705 View commit details
    Browse the repository at this point in the history

  7. nasbackup: implement restore backup 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    1d7b2c0 View commit details
    Browse the repository at this point in the history

  8. fixes 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    da5fe8d View commit details
    Browse the repository at this point in the history

  9. fix issues 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    e765ddb View commit details
    Browse the repository at this point in the history

  10. Update tools/marvin/setup.py

    @rohityadavcloud
    rohityadavcloud authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    5c3894d View commit details
    Browse the repository at this point in the history

  11. Update ui/src/views/AutogenView.vue

    @rohityadavcloud
    rohityadavcloud authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    e51dedf View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. fix pool ID issue in restored volume 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    c27d1ca View commit details
    Browse the repository at this point in the history

  2. fix potential NPE 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    8b6bb80 View commit details
    Browse the repository at this point in the history

  3. Update ui/src/components/view/ListView.vue 

    Co-authored-by: Abhishek Kumar <[email protected]>
    @rohityadavcloud @shwstppr
    rohityadavcloud and shwstppr authored Aug 29, 2024
    Configuration menu
    Copy the full SHA
    25dd780 View commit details
    Browse the repository at this point in the history

  4. Update server/src/main/java/org/apache/cloudstack/backup/BackupManage… 

    …rImpl.java

Co-authored-by: Abhishek Kumar <[email protected]>
    @rohityadavcloud @shwstppr
    rohityadavcloud and shwstppr authored Aug 29, 2024
    Configuration menu
    Copy the full SHA
    a1e1980 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. fix unused import

    @shwstppr
    shwstppr authored Aug 30, 2024
    Configuration menu
    Copy the full SHA
    0dc00d1 View commit details
    Browse the repository at this point in the history

  2. Update plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm… 

    …/resource/wrapper/LibvirtRestoreBackupCommandWrapper.java

Co-authored-by: Abhishek Kumar <[email protected]>
    @rohityadavcloud @shwstppr
    rohityadavcloud and shwstppr authored Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c09a977 View commit details
    Browse the repository at this point in the history

  3. Update core/src/main/java/org/apache/cloudstack/backup/DeleteBackupCo… 

    …mmand.java

Co-authored-by: Abhishek Kumar <[email protected]>
    @rohityadavcloud @shwstppr
    rohityadavcloud and shwstppr authored Aug 30, 2024
    Configuration menu
    Copy the full SHA
    26e48cf View commit details
    Browse the repository at this point in the history

  4. remove/fix weird spacing and newline 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    958049d View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/main' into kvm-backup-plugin-sha… 

    …red-storage
    @rohityadavcloud
    rohityadavcloud committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    5b68e9c View commit details
    Browse the repository at this point in the history

  6. Missing import

    @Pearl1594
    Pearl1594 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c50ca68 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. Fix created time to after the answer is recieved from the host and ad… 

    …d BackingUp and Failed state
    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    93cb33a View commit details
    Browse the repository at this point in the history

  2. fix lint

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    1ee025e View commit details
    Browse the repository at this point in the history

  3. Allow multiple backup schedules to be created

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    8f04bec View commit details
    Browse the repository at this point in the history

  4. update setup.py file

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    a476a94 View commit details
    Browse the repository at this point in the history

  5. fix linter

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    4764327 View commit details
    Browse the repository at this point in the history

  6. Perform virsh attach-disk operation for running VMs when restore volu… 

    …me to VM is performed
    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    37316f3 View commit details
    Browse the repository at this point in the history

  7. fix linter

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    f632820 View commit details
    Browse the repository at this point in the history

  8. address comments

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    9011353 View commit details
    Browse the repository at this point in the history

  9. cleanup restore op from script as its done via Java - using rsync

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    d3450b9 View commit details
    Browse the repository at this point in the history

  10. fix unit test

    @Pearl1594
    Pearl1594 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    5bd9eac View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. 1. adds backed_volumes column to the backups table to track volumes b… 

    …acked per backup object

2. remove device ID from backedup volume names to prevent issues when device ID changes
3. remove cifs and cephfs from the UI
    @Pearl1594
    Pearl1594 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    7e94bfa View commit details
    Browse the repository at this point in the history

  2. remove unused import

    @Pearl1594
    Pearl1594 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    ae9634d View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Allow attaching volumes to a VM with backup offering but with No backups

    @Pearl1594
    Pearl1594 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    1c28d98 View commit details
    Browse the repository at this point in the history

  2. Prevent deletion of backup repository if it contains backups

    @Pearl1594
    Pearl1594 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e065b4e View commit details
    Browse the repository at this point in the history

  3. Prevent normal user's from seeing offering that doesnt allow user dri… 

    …ven backups
    @Pearl1594
    Pearl1594 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    414943b View commit details
    Browse the repository at this point in the history

  4. fix unit test

    @Pearl1594
    Pearl1594 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    b7d64e2 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. address comments provided - rename variable and add some defensive co… 

    …de for backward compatibility - remove unused import
    @Pearl1594
    Pearl1594 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    fbe3c18 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into kvm-backup-plugin-shared-storage

    @rohityadavcloud
    rohityadavcloud authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    265466d View commit details
    Browse the repository at this point in the history

  3. Update schema-41910to42000.sql

    @rohityadavcloud
    rohityadavcloud authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    7213eb2 View commit details
    Browse the repository at this point in the history

  4. Update tools/marvin/setup.py

    @rohityadavcloud
    rohityadavcloud authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    8476fd4 View commit details
    Browse the repository at this point in the history

  5. Update engine/schema/src/main/resources/META-INF/db/schema-41910to420… 

    …00.sql

Co-authored-by: Suresh Kumar Anaparti <[email protected]>
    @rohityadavcloud @sureshanaparti
    rohityadavcloud and sureshanaparti authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    ee0c5b3 View commit details
    Browse the repository at this point in the history

  6. Update engine/schema/src/main/resources/META-INF/db/schema-41910to420… 

    …00.sql

Co-authored-by: Suresh Kumar Anaparti <[email protected]>
    @rohityadavcloud @sureshanaparti
    rohityadavcloud and sureshanaparti authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    42e7319 View commit details
    Browse the repository at this point in the history

  7. add rsync as dependnecy for cloudstack-agent 

    Signed-off-by: Rohit Yadav <[email protected]>
(cherry picked from commit ecfeb04026e1bbf13dc22bb0aa713677c7708bb8)
Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    9bffa4d View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'origin/main' into kvm-backup-plugin-sha… 

    …red-storage
    @rohityadavcloud
    rohityadavcloud committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    11c369e View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into kvm-backup-plugin-shared-storage

    @rohityadavcloud
    rohityadavcloud authored Sep 5, 2024
    Configuration menu
    Copy the full SHA
    7423f5f View commit details
    Browse the repository at this point in the history

  10. fix db schema setup issue 

    Signed-off-by: Rohit Yadav <[email protected]>
    @rohityadavcloud
    rohityadavcloud committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    9efdb57 View commit details
    Browse the repository at this point in the history