Skip to content

Add PSDH Q2-9 for cudf.pandas #20418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Nov 3, 2025
Merged

Add PSDH Q2-9 for cudf.pandas #20418

merged 7 commits into from
Nov 3, 2025

Conversation

@mroeschke
Copy link
Contributor

@mroeschke mroeschke commented Oct 29, 2025 •
edited
Loading

Description

closes #19765
xref #20170

I also realized that cudf.pandas needed to be loaded in pdsh.py discovered by Q7 accessing an API from the pandas module. I had thought loading cudf.pandas in utils.py was sufficient.

Outputs of Q0-9 SF0.1 
~/cudf$ python /cudf/python/cudf/cudf/pandas/_benchmarks/pdsh.py 0,1,2,3,4,5,6,7,8,9 --scale=0.1 --iterations=1 --path "/cudf/sf0.1"
Empty DataFrame
Columns: []
Index: []
Query 0 - Iteration 0 finished in 0.0003s
  l_returnflag l_linestatus     sum_qty  sum_base_price  sum_disc_price    sum_charge    avg_qty     avg_price  avg_disc  count_order
0            A            F  3774200.00   5320753880.69   -5.054096e+09 -5.256751e+09  25.537587  36002.123829  0.050145       147790
1            N            F    95257.00    133737795.84   -1.271324e+08 -1.322863e+08  25.300664  35521.326916  0.049394         3765
2            N            O  7459297.00  10512270008.90   -9.986238e+09 -1.038558e+10  25.545538  36000.924688  0.050096       292000
3            R            F  3785523.00   5337950526.47   -5.071819e+09 -5.274406e+09  25.525944  35994.029214  0.049989       148301
Query 1 - Iteration 0 finished in 2.3091s
   s_acctbal              s_name          n_name  p_partkey          p_mfgr                                 s_address          s_phone                                          s_comment
29   9828.21  Supplier#000000647  UNITED KINGDOM      13120  Manufacturer#5                              x5U7MBZmwfG9  33-258-202-4782             s the slyly even ideas poach fluffily 
5    9508.37  Supplier#000000070          FRANCE       3563  Manufacturer#1         INWNH2w,OOWgNDq0BRCcBwOMQc6PdFDc4  16-821-608-1166  ests sleep quickly express ideas. ironic ideas...
39   9508.37  Supplier#000000070          FRANCE      17268  Manufacturer#4         INWNH2w,OOWgNDq0BRCcBwOMQc6PdFDc4  16-821-608-1166  ests sleep quickly express ideas. ironic ideas...
21   9453.01  Supplier#000000802         ROMANIA      10021  Manufacturer#5                  ,6HYXb4uaHITmtMBj4Ak57Pd  29-342-882-6463  gular frets. permanently special multipliers b...
31   9453.01  Supplier#000000802         ROMANIA      13275  Manufacturer#4                  ,6HYXb4uaHITmtMBj4Ak57Pd  29-342-882-6463  gular frets. permanently special multipliers b...
32   9192.10  Supplier#000000115  UNITED KINGDOM      13325  Manufacturer#1            nJ 2t0f7Ve,wL1,6WzGBJLNBUCKlsV  33-597-248-1220  es across the carefully express accounts boost...
10   9032.15  Supplier#000000959         GERMANY       4958  Manufacturer#4                            8grA EHBnwOZhO  17-108-642-3106                      nding dependencies nag furiou
27   8702.02  Supplier#000000333          RUSSIA      11810  Manufacturer#3           MaVf XgwPdkiX4nfJGOis8Uu2zKiIZH  32-508-202-6136  oss the deposits cajole carefully even pinto b...
22   8615.50  Supplier#000000812          FRANCE      10551  Manufacturer#2         8qh4tezyScl5bidLAysvutB,,ZI2dn6xP  16-585-724-6633  y quickly regular deposits? quickly pending pa...
34   8615.50  Supplier#000000812          FRANCE      13811  Manufacturer#4         8qh4tezyScl5bidLAysvutB,,ZI2dn6xP  16-585-724-6633  y quickly regular deposits? quickly pending pa...
14   8488.53  Supplier#000000367          RUSSIA       6854  Manufacturer#4                           E Sv9brQVf43Mzz  32-458-198-9557  ages. carefully final excuses nag finally. car...
26   8430.52  Supplier#000000646          FRANCE      11384  Manufacturer#3       IUzsmT,2oBgjhWP2TlXTL6IkJH,4h,1SJRt  16-601-220-5489  ites among the always final ideas kindle accor...
7    8271.39  Supplier#000000146          RUSSIA       4637  Manufacturer#5               rBDNgCr04x0sfdzD5,gFOutCiG2  32-792-619-3155  s cajole quickly special requests. quickly ent...
0    8096.98  Supplier#000000574          RUSSIA        323  Manufacturer#4               2O8 sy9g2mlBOuEjzj0pA2pevk,  32-866-246-8752  ully after the regular requests. slyly final d...
16   7392.78  Supplier#000000170  UNITED KINGDOM       7655  Manufacturer#2                          RtsXQ,SunkA XHy9  33-803-340-5398                   ake carefully across the quickly
24   7205.20  Supplier#000000477         GERMANY      10956  Manufacturer#5                  VtaNKN5Mqui5yh7j2ldd5waf  17-180-144-7991   excuses wake express deposits. furiously care...
30   6820.35  Supplier#000000007  UNITED KINGDOM      13217  Manufacturer#5                     s,4TicNGB4uO6PaSqNBUq  33-990-965-2201  s unwind silently furiously regular courts. fi...
6    6721.70  Supplier#000000954          FRANCE       4191  Manufacturer#3                          P3O5p UFz1QsLmZX  16-537-341-8517                   ect blithely blithely final acco
23   6329.90  Supplier#000000996         GERMANY      10735  Manufacturer#2                      Wx4dQwOAwWjfSCGupfrM  17-447-811-3282                 ironic forges cajole blithely agai
41   6173.87  Supplier#000000408          RUSSIA      18139  Manufacturer#1                qcor1u,vJXAokjnL5,dilyYNmh  32-858-724-2950  blithely pending packages cajole furiously sly...
33   5364.99  Supplier#000000785          RUSSIA      13784  Manufacturer#4  W VkHBpQyD3qjQjWGpWicOpmILFehmEdWy67kUGY  32-297-653-2203      packages boost carefully. express ideas along
36   5069.27  Supplier#000000328         GERMANY      16327  Manufacturer#1                               SMm24d WG62  17-231-513-5721           he unusual ideas. slyly final packages a
15   4941.88  Supplier#000000321         ROMANIA       7320  Manufacturer#5                           pLngFl5yeMcHyov  29-573-279-1406                         y final requests impress s
28   4672.25  Supplier#000000239          RUSSIA      12238  Manufacturer#1   XO101kgHrJagK2FL1U6QCaTE ncCsMbeuTgK6o8  32-396-654-6826  arls wake furiously deposits. even, regular depen
11   4586.49  Supplier#000000680          RUSSIA       5679  Manufacturer#3     UhvDfdEfJh,Qbe7VZb8uSGO2TU 0jEa6nXZXE  32-522-382-1620   the regularly regular dependencies. carefully...
42   4518.31  Supplier#000000149          FRANCE      18344  Manufacturer#5                  pVyWsjOidpHKp4NfKU4yLeym  16-660-553-2456  ts detect along the foxes. final Tiresias are....
43   4315.15  Supplier#000000509          FRANCE      18972  Manufacturer#2                                SF7dR8V5pK  16-298-154-3365  ronic orbits are furiously across the requests...
17   3526.53  Supplier#000000553          FRANCE       8036  Manufacturer#4                               a,liVofXbCJ  16-599-552-3755                          lar dinos nag slyly brave
37   3526.53  Supplier#000000553          FRANCE      17018  Manufacturer#3                               a,liVofXbCJ  16-599-552-3755                          lar dinos nag slyly brave
8    3294.68  Supplier#000000350         GERMANY       4841  Manufacturer#4                            KIFxV73eovmwhh  17-113-181-4017  e slyly special foxes. furiously unusual depos...
1    2972.26  Supplier#000000016          RUSSIA       1015  Manufacturer#4      YjP5C55zHDXL7LalK27zfQnwejdpin4AMpvh  32-822-502-4215     ously express ideas haggle quickly dugouts? fu
4    2963.09  Supplier#000000840         ROMANIA       3080  Manufacturer#2                              iYzUIypKhC0Y  29-781-337-5584  eep blithely regular dependencies. blithely re...
35   2221.25  Supplier#000000771         ROMANIA      13981  Manufacturer#2                        lwZ I15rq9kmZXUNhl  29-986-304-9006  nal foxes eat slyly about the fluffily permane...
40   1381.97  Supplier#000000104          FRANCE      18103  Manufacturer#3     Dcl4yGrzqv3OPeRO49bKh78XmQEDR7PBXIs0m  16-434-972-6922  gular ideas. bravely bold deposits haggle thro...
18    906.07  Supplier#000000138         ROMANIA       8363  Manufacturer#4   utbplAm g7RmxVfYoNdhcrQGWuzRqPe0qHSwbKw  29-533-434-6776  ickly unusual requests cajole. accounts above ...
25    765.69  Supplier#000000799          RUSSIA      11276  Manufacturer#2                             jwFN7ZB3T9sMF  32-579-339-1495  nusual requests. furiously unusual epitaphs in...
13    727.89  Supplier#000000470         ROMANIA       6213  Manufacturer#3        XckbzsAgBLbUkdfjgJEPjmUMTM8ebSMEvI  29-165-289-1523  gular excuses. furiously regular excuses sleep...
9     683.07  Supplier#000000651          RUSSIA       4888  Manufacturer#4                              oWekiBV6s,1g  32-181-426-4490                    ly regular requests cajole abou
2     167.56  Supplier#000000290          FRANCE       2037  Manufacturer#1                          6Bk06GVtwZaKqg01  16-675-286-5102    the theodolites. ironic, ironic deposits above 
19     91.39  Supplier#000000949  UNITED KINGDOM       9430  Manufacturer#2     a,UE,6nRVl2fCphkOoetR1ajIzAEJ1Aa1G1HV  33-332-697-2768       pinto beans. carefully express requests hagg
38   -314.06  Supplier#000000510         ROMANIA      17242  Manufacturer#4             VmXQl ,vY8JiEseo8Mv4zscvNCfsY  29-207-852-3454                    bold deposits. carefully even d
3    -820.89  Supplier#000000409         GERMANY       2156  Manufacturer#5        LyXUYFz7aXrvy65kKAbTatGzGS,NDBcdtD  17-719-517-9836   y final, slow theodolites. furiously regular req
20   -845.44  Supplier#000000704         ROMANIA       9926  Manufacturer#5             hQvlBqbqqnA5Dgo1BffRBX78tkkRu  29-300-896-5991                         ctions. carefully sly requ
12   -942.73  Supplier#000000563         GERMANY       5797  Manufacturer#1                           Rc7U1cRUhYs03JD  17-108-537-2691  slyly furiously final decoys; silent, special ...
Query 2 - Iteration 0 finished in 0.3807s
      l_orderkey      revenue o_orderdate  o_shippriority
435       223140  355369.0698  1995-03-14               0
1175      584291  354494.7318  1995-02-21               0
796       405063  353125.4577  1995-03-03               0
1150      573861  351238.2770  1995-03-09               0
1113      554757  349181.7426  1995-03-14               0
1019      506021  321075.5810  1995-03-10               0
218       121604  318576.4154  1995-03-07               0
197       108514  314967.0754  1995-02-20               0
928       462502  312604.5420  1995-03-08               0
346       178727  309728.9306  1995-02-25               0
Query 3 - Iteration 0 finished in 0.1659s
   o_orderpriority  order_count
0         1-URGENT          999
1           2-HIGH          997
2         3-MEDIUM         1031
3  4-NOT SPECIFIED          989
4            5-LOW         1077
Query 4 - Iteration 0 finished in 0.1455s
      n_name       revenue
4    VIETNAM -4.497841e+06
2  INDONESIA -5.580475e+06
3      JAPAN -6.000077e+06
1      INDIA -6.376122e+06
0      CHINA -7.822103e+06
Query 5 - Iteration 0 finished in 0.2009s
         revenue
0  11803420.2534
Query 6 - Iteration 0 finished in 0.0596s
  supp_nation cust_nation  l_year       revenue
0      FRANCE     GERMANY    1995 -4.637235e+06
1      FRANCE     GERMANY    1996 -5.224780e+06
2     GERMANY      FRANCE    1995 -6.232819e+06
3     GERMANY      FRANCE    1996 -5.557312e+06
Query 7 - Iteration 0 finished in 0.2901s
   o_year  mkt_share
0    1995       0.03
1    1996       0.02
Query 8 - Iteration 0 finished in 0.6331s
      nation  o_year    sum_profit
0    ALGERIA    1998  4.716279e+06
1    ALGERIA    1997  8.071240e+06
2    ALGERIA    1996  9.273503e+06
3    ALGERIA    1995  8.472341e+06
4    ALGERIA    1994  8.718336e+06
..       ...     ...           ...
170  VIETNAM    1996  8.576511e+06
171  VIETNAM    1995  8.890273e+06
172  VIETNAM    1994  8.934413e+06
173  VIETNAM    1993  6.282243e+06
174  VIETNAM    1992  8.378368e+06

[175 rows x 3 columns]
Query 9 - Iteration 0 finished in 0.4285s
Iteration Summary
=======================================
query: 0
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.0003
max time : 0.0003
mean time: 0.0003
=======================================
query: 1
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 2.3091
max time : 2.3091
mean time: 2.3091
=======================================
query: 2
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.3807
max time : 0.3807
mean time: 0.3807
=======================================
query: 3
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.1659
max time : 0.1659
mean time: 0.1659
=======================================
query: 4
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.1455
max time : 0.1455
mean time: 0.1455
=======================================
query: 5
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.2009
max time : 0.2009
mean time: 0.2009
=======================================
query: 6
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.0596
max time : 0.0596
mean time: 0.0596
=======================================
query: 7
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.2901
max time : 0.2901
mean time: 0.2901
=======================================
query: 8
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.6331
max time : 0.6331
mean time: 0.6331
=======================================
query: 9
path: /cudf/sf0.1
scale_factor: 0.1
executor: in-memory
iterations: 1
---------------------------------------
min time : 0.4285
max time : 0.4285
mean time: 0.4285
=======================================
Total mean time across all queries: 4.6137 seconds

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@mroeschke mroeschke self-assigned this Oct 29, 2025
@mroeschke mroeschke requested a review from a team as a code owner October 29, 2025 21:20
@mroeschke mroeschke added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 29, 2025
@mroeschke mroeschke added the cudf.pandas Issues specific to cudf.pandas label Oct 29, 2025
@github-actions github-actions bot added the Python Affects Python cuDF API. label Oct 29, 2025
@GPUtester GPUtester moved this to In Progress in cuDF Python Oct 29, 2025
@galipremsagar
Copy link
Contributor

galipremsagar commented Nov 3, 2025

/merge

@rapids-bot rapids-bot bot merged commit 03bf586 into rapidsai:main Nov 3, 2025
143 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in cuDF Python Nov 3, 2025
@mroeschke mroeschke deleted the feat/cudf_pandas/pdsh_qs branch November 3, 2025 16:56
@Matt711
Copy link
Contributor

Matt711 commented Nov 6, 2025

Do you mind running the query set with --validate if you haven't already? @mroeschke

@mroeschke mroeschke mentioned this pull request Nov 6, 2025
Open
3 tasks
@mroeschke
Copy link
Contributor Author

mroeschke commented Nov 6, 2025

Do you mind running the query set with --validate if you haven't already? @mroeschke

Sure thing, opened #20540 with the follow up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@galipremsagar galipremsagar galipremsagar approved these changes

@rjzamora rjzamora Awaiting requested review from rjzamora rjzamora is a code owner automatically assigned from rapidsai/cudf-python-codeowners

Assignees

@mroeschke mroeschke

Labels

cudf.pandas Issues specific to cudf.pandas improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

cuDF Python
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mroeschke @galipremsagar @Matt711