Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(llmobs): add sampling for ragas skeleton code #10719

Merged
merged 60 commits into from
Oct 9, 2024
Merged

chore(llmobs): add sampling for ragas skeleton code #10719

merged 60 commits into from
Oct 9, 2024

Commits on Sep 13, 2024

  1. implement ragas faithfulenss runner with dummy ragas score generator

    lievan committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    571d317 View commit details
    Browse the repository at this point in the history

  2. remove newline

    lievan committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    4b3d840 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. pydantic v1

    lievan committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    7b9c929 View commit details
    Browse the repository at this point in the history

  2. refactor into evaluator list

    lievan committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    2e883a0 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. add unit tests

    lievan committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7b31443 View commit details
    Browse the repository at this point in the history

  2. fix expectde span event

    lievan committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    13229bd View commit details
    Browse the repository at this point in the history

  3. merg conf

    lievan committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    b493e20 View commit details
    Browse the repository at this point in the history

  4. remove config option, use only env var

    lievan committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    d290dcd View commit details
    Browse the repository at this point in the history

  5. address comments

    lievan committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6e49cca View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. refactor into one evaluator service

    lievan committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    fcf9991 View commit details
    Browse the repository at this point in the history

  2. dont cancel futures

    lievan committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    10b276f View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. refactor dummy faithfulness into class

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    b6fa4e0 View commit details
    Browse the repository at this point in the history

  2. sampler

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    60d4e5c View commit details
    Browse the repository at this point in the history

  3. sampler wip

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    064b770 View commit details
    Browse the repository at this point in the history

  4. fix tests'

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    cd478bf View commit details
    Browse the repository at this point in the history

  5. refactor into seperate file

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    3910c40 View commit details
    Browse the repository at this point in the history

  6. rename field to label

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    be893a1 View commit details
    Browse the repository at this point in the history

  7. change sampling rules parsing

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    7044405 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into evan.li/ragas-skeleton

    @lievan
    lievan authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    a309330 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' of github.com:DataDog/dd-trace-py into evan.li/ra… 

    …gas-skeleton
    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    d849067 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'evan.li/ragas-skeleton' of github.com:DataDog/dd-trace-… 

    …py into evan.li/ragas-skeleton
    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    fd73621 View commit details
    Browse the repository at this point in the history

  11. dont use self.choose_matcher

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    64df97b View commit details
    Browse the repository at this point in the history

  12. think about rate limiting

    lievan committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    f5b6518 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. wip tests

    lievan committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    22cb049 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:DataDog/dd-trace-py into evan.li/ra… 

    …gas-skeleton
    lievan committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2f48461 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into evan.li/ragas-skeleton

    @lievan
    lievan authored Sep 20, 2024
    Configuration menu
    Copy the full SHA
    04d202e View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. refactor so we store the service only in ragas

    lievan committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    a991f15 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'evan.li/ragas-skeleton' of github.com:DataDog/dd-trace-… 

    …py into evan.li/ragas-skeleton
    lievan committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    38e0a23 View commit details
    Browse the repository at this point in the history

  3. rename a test

    lievan committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    ea5d4fa View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. clean up

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e66ee7e View commit details
    Browse the repository at this point in the history

  2. rename, fix test

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    c04dac4 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:DataDog/dd-trace-py into evan.li/ra… 

    …gas-skeleton
    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0a956e1 View commit details
    Browse the repository at this point in the history

  4. fork safety

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    6d9c136 View commit details
    Browse the repository at this point in the history

  5. fix tests

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7d7192c View commit details
    Browse the repository at this point in the history

  6. add more comments

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    bb8d388 View commit details
    Browse the repository at this point in the history

  7. delete unused fixture

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    3c17dee View commit details
    Browse the repository at this point in the history

  8. remove debug

    lievan committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    62ecbe4 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. merge latest skeleton

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    4848304 View commit details
    Browse the repository at this point in the history

  2. add back in sampling

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    bec14bf View commit details
    Browse the repository at this point in the history

  3. fix tests

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    268230c View commit details
    Browse the repository at this point in the history

  4. fix the rules

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    8f416bc View commit details
    Browse the repository at this point in the history

  5. change parsing error msg

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    177e7f3 View commit details
    Browse the repository at this point in the history

  6. fix test

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    8e4b376 View commit details
    Browse the repository at this point in the history

  7. add comment for default sampling

    lievan committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    3ff78f7 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. fix wrong patch

    lievan committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    f43dceb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'evan.li/ragas-skeleton' into evan.li/ragas-skeleton-wit… 

    …h-sampling
    lievan committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    0cf3312 View commit details
    Browse the repository at this point in the history

  3. update key names for sampling rules

    lievan committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    6b3c301 View commit details
    Browse the repository at this point in the history

  4. merge conf

    lievan committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    b932e96 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. don't try to handle decoder errors

    lievan committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    e9fdec6 View commit details
    Browse the repository at this point in the history

  2. rm some extra lines

    lievan committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    7a85591 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. update type annotations

    lievan committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    16c368b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:DataDog/dd-trace-py into evan.li/ra… 

    …gas-skeleton-with-sampling
    lievan committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    c11d058 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into evan.li/ragas-skeleton-with-sampling

    @lievan
    lievan authored Oct 4, 2024
    Configuration menu
    Copy the full SHA
    c4cdb32 View commit details
    Browse the repository at this point in the history

  4. union type

    lievan committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    4c699f3 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'evan.li/ragas-skeleton-with-sampling' of github.com:Dat… 

    …aDog/dd-trace-py into evan.li/ragas-skeleton-with-sampling
    lievan committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    86b1e27 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. soft fail for invalid floats

    lievan committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    18a7dae View commit details
    Browse the repository at this point in the history

  2. rm newline

    lievan committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    8e521cb View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:DataDog/dd-trace-py into evan.li/ra… 

    …gas-skeleton-with-sampling
    lievan committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d0718a1 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. add a dummy evaluator and a casette

    lievan committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    098d8fa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into evan.li/ragas-skeleton-with-sampling

    @lievan
    lievan authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    2ae2917 View commit details
    Browse the repository at this point in the history