Skip to content

cudaErrorMemoryAllocation w/ TZ basis sets #746

@corinwagen

Description

@corinwagen

I'm having issues with OOM errors when I run ωB97M-V/def2-TZVP/CPCM(THF) calculations on large systems. Even on an H200, I often get cudaErrorMemoryAllocation errors. Is there a setting that I can tune to get this to run within the available VRAM, or is it simply too big for GPU4PySCF at this time (with this combination of functional and basis set)?

Here's the error:

  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/scf/hf.py", line 356, in scf
    _kernel(mf, mf.conv_tol, mf.conv_tol_grad,
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/scf/hf.py", line 214, in _kernel
    vhf = mf.get_veff(mol, dm)
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/solvent/_attach_solvent.py", line 78, in get_veff
    veff = super().get_veff(mol, dm_or_wfn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df_jk.py", line 270, in get_veff
    vj, vk = self.get_jk(mol, dm, hermi)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df_jk.py", line 135, in get_jk
    vj, vk = self.with_df.get_jk(dm, hermi, with_j, with_k,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df.py", line 140, in get_jk
    return df_jk.get_jk(self, dm, hermi, with_j, with_k, direct_scf_tol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df_jk.py", line 573, in get_jk
    dfobj.build(direct_scf_tol=direct_scf_tol, omega=omega)
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df.py", line 130, in build
    self._cderi = cholesky_eri_gpu(intopt, mol, auxmol, self.cd_low,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/gpu4pyscf/df/df.py", line 259, in cholesky_eri_gpu
    mem = cupy.cuda.alloc_pinned_memory((p1-p0) * npairs * 8)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cupy/cuda/pinned_memory.pyx", line 201, in cupy.cuda.pinned_memory.alloc_pinned_memory
  File "cupy/cuda/pinned_memory.pyx", line 215, in cupy.cuda.pinned_memory.alloc_pinned_memory
  File "cupy/cuda/pinned_memory.pyx", line 289, in cupy.cuda.pinned_memory.PinnedMemoryPool.malloc
  File "cupy/cuda/pinned_memory.pyx", line 311, in cupy.cuda.pinned_memory.PinnedMemoryPool.malloc
  File "cupy/cuda/pinned_memory.pyx", line 180, in cupy.cuda.pinned_memory._malloc
  File "cupy/cuda/pinned_memory.pyx", line 181, in cupy.cuda.pinned_memory._malloc
  File "cupy/cuda/pinned_memory.pyx", line 30, in cupy.cuda.pinned_memory.PinnedMemory.__init__
  File "cupy_backends/cuda/api/runtime.pyx", line 584, in cupy_backends.cuda.api.runtime.hostAlloc
  File "cupy_backends/cuda/api/runtime.pyx", line 146, in cupy_backends.cuda.api.runtime.check_status
cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorMemoryAllocation: out of memory

We're using density fitting, 0.10 level shift, Becke grids, etc. Pretty standard settings. Here's the structure; it's a fairly routine hydrocupration with a chiral diphosphine ligand:

197
name: Single point structure; charge: 0; multiplicity: 1; energy: -5775.23695400; method: ωB97M-V; basis_set: def2-tzvp; engine: GPU4PySCF; generated_by: Rowan; timestamp: 2026-04-27 15:29:46;
Cu   -0.00118995   0.04142683  -1.65858433
H     0.40472948  -1.18275635  -2.65283272
C    -0.28728372   1.29424052  -3.29662903
C     0.16165144   0.02861187  -3.77605366
P     1.64585506   0.45479689  -0.15510687
C     0.99750937   1.33305806   1.32205690
C     3.01600818   1.48958215  -0.76325366
C     2.49350334  -0.99981811   0.52068412
C     0.10215160   0.69440513   2.18946768
C     1.30625388   2.68069049   1.51792651
C     4.34397034   1.15066646  -0.60917282
C     2.69810794   2.63375968  -1.49460667
C     2.36647834  -2.18182340  -0.17907921
C     3.29225252  -0.96486780   1.66098871
C    -0.44280600   1.42992485   3.25308908
C    -0.23682219  -0.76486058   2.14436119
C     0.73587743   3.39016893   2.55743777
H     2.00591631   3.17568950   0.85922755
C     5.37677334   1.87710106  -1.20725284
H     4.59486698   0.28073707  -0.02067536
C     3.67119594   3.39787488  -2.11741172
H     1.65518811   2.90727658  -1.58224118
C     3.07628790  -3.33281189   0.16812237
H     1.68965496  -2.19379168  -1.02561850
C     4.06627441  -2.05496481   2.02818412
H     3.31030510  -0.05882880   2.25121179
C    -0.14383827   2.77160143   3.43186272
O    -1.27632458   0.74895456   4.09302247
C     0.27208481  -1.55787496   3.18298703
C    -1.09538187  -1.35641718   1.20900004
H     0.98214717   4.43567336   2.69605380
C     5.00355716   2.93594065  -2.03925294
C     4.00447673  -3.20251687   1.20552875
C    -1.39744048   1.24038507   5.41521032
C    -0.03462244  -2.90635383   3.27875220
O     1.08480111  -0.92975946   4.08376986
C    -1.40267141  -2.71410808   1.31276762
P    -1.69498319  -0.41085296  -0.24370938
H    -1.98530105   2.15953938   5.46222188
H    -1.91454719   0.46833864   5.98131658
C    -0.87534953  -3.47815794   2.33701935
C     1.08257629  -1.43505968   5.40675556
H    -2.06306532  -3.17317257   0.58932656
C    -3.01471387  -1.44826790  -0.94765405
C    -2.58559419   1.00144767   0.46401603
H    -1.12165261  -4.53043661   2.40739091
H     0.05888985  -1.56756312   5.76734076
H     1.59462926  -0.69408948   6.01750648
C    -4.33855285  -1.38184863  -0.56374105
C    -2.64604213  -2.35132412  -1.94119322
C    -2.45293029   2.20554180  -0.19710819
C    -3.40786462   0.92799721   1.58529246
C    -5.33232554  -2.14554232  -1.17768768
H    -4.61907988  -0.70197931   0.22709574
C    -3.58218232  -3.13445296  -2.59926525
H    -1.59769437  -2.40733728  -2.20424480
C    -3.18406225   3.33803005   0.16436485
H    -1.75195468   2.25081462  -1.02317554
C    -4.21154829   1.99520534   1.95646451
H    -3.41673470   0.01231090   2.16095195
C    -4.93771001  -2.94218520  -2.25959085
C    -4.14462085   3.16430449   1.16618799
C     4.89279182  -2.02876812   3.32410547
C     4.41988483  -3.17301649   4.23349694
H     4.60090971  -4.14397035   3.77694437
H     4.94821623  -3.13383701   5.18942699
H     3.35012856  -3.07916391   4.42875392
C     6.40212640  -2.16680190   3.07635266
H     6.93883729  -1.96610071   4.00660043
H     6.67124879  -3.16777287   2.75081227
H     6.74827244  -1.44519979   2.33231374
C     4.68168111  -0.71355250   4.08271885
H     5.06688644   0.14367130   3.52515035
H     3.62787206  -0.53588843   4.30178017
H     5.22488669  -0.76232817   5.02826767
C     2.72168261  -4.61290538  -0.61185465
C     3.16680224  -4.47434178  -2.07467466
H     4.25034927  -4.37102409  -2.15857030
H     2.86716917  -5.36001449  -2.64012395
H     2.70870311  -3.60183877  -2.54419909
C     1.18745770  -4.76480861  -0.58623861
H     0.82732406  -4.84051423   0.44134811
H     0.67255284  -3.92797338  -1.05848873
H     0.90522729  -5.67509325  -1.12050130
C     3.27863801  -5.91680741  -0.02789283
H     4.35337228  -6.02260668  -0.14866119
H     3.04692067  -6.00890319   1.03330713
H     2.80838645  -6.75161283  -0.55231141
C    -3.12658011  -4.20569292  -3.60290354
C    -3.63216244  -3.94432705  -5.02874330
H    -4.70332152  -4.10456370  -5.11596341
H    -3.39572954  -2.92728525  -5.34988040
H    -3.13836433  -4.63374093  -5.71722110
C    -1.59650713  -4.27891155  -3.67383965
H    -1.15381135  -3.35639297  -4.05615125
H    -1.15343312  -4.49112484  -2.69900711
H    -1.31636542  -5.08815737  -4.35060603
C    -3.62893839  -5.57686563  -3.12415766
H    -3.29729286  -6.35423068  -3.81686113
H    -3.22202941  -5.80800463  -2.13710610
H    -4.71531716  -5.60612283  -3.06797863
C    -6.74334038  -2.04397471  -0.56401795
C    -7.74809890  -3.10351165  -1.03363723
H    -8.63506530  -3.03480928  -0.39993198
H    -8.07305998  -2.96749186  -2.06101180
H    -7.34021749  -4.10965771  -0.93906625
C    -7.32131089  -0.64389665  -0.81788896
H    -6.66214467   0.13206174  -0.42219872
H    -7.45765183  -0.45167725  -1.88337398
H    -8.29380002  -0.54650348  -0.32895479
C    -6.61483563  -2.24135786   0.96021243
H    -6.16110124  -3.20785593   1.18772160
H    -6.01477559  -1.46634428   1.43592104
H    -7.60711742  -2.21300713   1.41495839
C    -2.81498781   4.64930654  -0.55408684
C    -1.28243055   4.80320486  -0.48195933
H    -0.94500794   4.81955447   0.55645677
H    -0.75474315   3.99707306  -0.99207959
H    -0.99019510   5.74218351  -0.95752039
C    -3.39270450   5.92471911   0.07115101
H    -4.46434009   6.03207795  -0.07309283
H    -3.18970469   5.97048361   1.14135007
H    -2.91233436   6.78335385  -0.40314477
C    -3.21816007   4.57312698  -2.03337498
H    -4.29828870   4.47424305  -2.15428255
H    -2.90206615   5.48130384  -2.55213732
H    -2.74600155   3.72154514  -2.52641509
C    -5.09155469   1.91275304   3.21405762
C    -4.72377947   3.06142223   4.16551317
H    -4.93280815   4.03124044   3.71984821
H    -5.29444156   2.97412462   5.09342123
H    -3.66122243   3.02074725   4.41282335
C    -4.85642771   0.59685481   3.96454821
H    -5.18867080  -0.26792581   3.38545729
H    -3.80534332   0.45482277   4.21786163
H    -5.43437253   0.61228916   4.89042942
C    -6.59021976   1.97831156   2.88585189
H    -7.16751043   1.74831395   3.78457054
H    -6.89016954   2.96566328   2.54567696
H    -6.85692568   1.24270515   2.12220778
C     6.80930841   1.44146398  -0.84126683
C     6.89545188   1.35071412   0.69603619
H     6.67109582   2.31731399   1.15144013
H     6.20928619   0.61416660   1.11351978
H     7.90767421   1.06198363   0.98739696
C     7.09597708   0.04979912  -1.42367307
H     6.36747967  -0.68434487  -1.07284036
H     7.06142804   0.05679215  -2.51448132
H     8.09054089  -0.28488429  -1.11854106
C     7.92405005   2.40693215  -1.26194377
H     8.85352401   2.07718991  -0.79232585
H     8.09373050   2.43044536  -2.33468250
H     7.71792897   3.42416712  -0.93003607
C     3.30336521   4.72525837  -2.79946031
C     1.82289886   5.06131283  -2.58844109
H     1.16071882   4.34467273  -3.07914691
H     1.56235547   5.10304154  -1.52893378
H     1.62196287   6.04393373  -3.01922412
C     3.55100036   4.70487679  -4.31423488
H     4.61133615   4.69939128  -4.55201908
H     3.07538697   3.83727431  -4.77762704
H     3.11685260   5.60167395  -4.76211910
C     4.12147923   5.85687663  -2.15723799
H     3.90327650   5.92521699  -1.08900796
H     5.19046882   5.69975863  -2.28402624
H     3.85501133   6.81127251  -2.61786424
O     4.90784781  -4.21794092   1.46010206
O     5.94573480   3.56119884  -2.83459210
O    -5.88922380  -3.56371891  -3.04661639
O    -5.07499217   4.15617217   1.41661239
C     6.23654178   2.80609127  -4.00515389
H     6.94934652   3.38744013  -4.58747801
H     6.67551619   1.83783670  -3.75631916
H     5.33353330   2.63417649  -4.59513574
C     5.98527804  -4.21639013   0.53159952
H     6.65702003  -5.02164020   0.82433887
H     5.63481034  -4.38923883  -0.48752339
H     6.52212044  -3.26453285   0.55655419
C    -6.11056030   4.17526827   0.44183722
H    -6.81877243   4.94348812   0.74781533
H    -5.72284421   4.41458017  -0.54971421
H    -6.61794704   3.20846785   0.39084709
C    -6.47611986  -2.66703380  -3.98249297
H    -7.17439818  -3.24836969  -4.58216695
H    -7.01415792  -1.86099792  -3.48000648
H    -5.71678223  -2.22371734  -4.63013198
C     1.55492429  -0.07605300  -4.35084635
H     2.26612883   0.44500893  -3.70570020
H     1.87789541  -1.11264613  -4.44550756
H     1.58245624   0.39518030  -5.33877653
H    -1.32865903   1.56230901  -3.43646887
H     0.42270285   2.11426665  -3.32387891
H    -0.57301037  -0.59865263  -4.26870520
H     0.37581036  -3.50847700   4.07746844
H     1.61720479  -2.38362850   5.49010495
H    -0.58422618   3.33206165   4.24443561
H    -0.41207666   1.42140187   5.85322833

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions