Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 26, 2025

Functions like compute_parquet() and compute_csv() would be useful for backends other than DuckDB (e.g., tidypolars). Converting them to S3 generics enables other packages to provide their own implementations.

Changes

  • compute_parquet() / compute_csv(): Converted to S3 generics using UseMethod()
  • .duckplyr_df methods: Contain the existing DuckDB-specific implementation
  • .data.frame methods: Convert input to duckplyr_df, then delegate to the duckplyr_df method
  • Tests: Added coverage for generic dispatch with both input types

Usage for other backends

# tidypolars or other packages can now implement:
compute_parquet.polars_df <- function(x, path, ...) {

  # backend-specific implementation
}

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • CRAN.R-project.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --vanilla --no-echo (dns block)
  • app.codecov.io
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • bioconductor.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --vanilla --no-echo (dns block)
  • cran.rstudio.com
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • dbplyr.tidyverse.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • dplyr.tidyverse.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • dtplyr.tidyverse.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • duckdb.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • duckplyr.tidyverse.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • extensions.duckdb.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --slave --no-save --no-restore -f /tmp/RtmpVHmxD3/callr-scr-2b3d34553bcf (dns block)
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --slave --no-save --no-restore -f /tmp/Rtmptz4Gfx/working_dir/RtmpiPUS0v/callr-scr-3319526a7cb5 (dns block)
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --slave --no-save --no-restore -f /tmp/Rtmptz4Gfx/working_dir/RtmpoNnMlg/callr-scr-36a764a1b446 (dns block)
  • forum.posit.co
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • lifecycle.r-lib.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • orcid.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • packagemanager.posit.co
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • r4ds.hadley.nz
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • ror.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • tidyverse.r-universe.dev
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)
  • usethis.r-lib.org
    • Triggering command: /opt/R/4.5.2/lib/R/bin/exec/R /opt/R/4.5.2/lib/R/bin/exec/R --no-restore --no-echo --args nextArgduckplyr_1.1.3.9004.tar.gznextArg--no-manualnextArg--as-cran (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Move compute_parquet etc. to upstream dplyr and make them generic functions</issue_title>
<issue_description>Functions like these would be useful for backends other than DuckDB, so I suspect there would be value in generic functions owned by dplyr.

For now, sink_parquet() of tidypolars currently does the same thing. (cc @etiennebacher)
https://github.com/etiennebacher/tidypolars/blob/d2b9d7b305880b48880c4cd476b27395cdd138dd/R/sink.R</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Move compute_parquet to upstream dplyr as generic function Make compute_parquet and compute_csv generic S3 functions Nov 26, 2025
Copilot AI requested a review from krlmlr November 26, 2025 10:35
Copilot finished work on behalf of krlmlr November 26, 2025 10:35
@etiennebacher
Copy link

Quick question on the implementation: does that mean tidypolars would have to import duckplyr to use the compute_*() methods? If so, that's probably not going to happen, which is why it'd be better if those functions were hosted in dplyr.

@krlmlr
Copy link
Member

krlmlr commented Nov 26, 2025

Hosting in dplyr is also not going to happen, as it seems. What other solutions do you propose?

@etiennebacher
Copy link

Ah, I didn't see the issue was closed in dplyr. I don't have a good alternative to propose then :/

@krlmlr
Copy link
Member

krlmlr commented Nov 26, 2025

One of the future goals of duckplyr is to offer a seamless and convenient interface with other systems.

If the generics lived here, tidypolars could still implement them without making duckplyr a strong dependency. Chances are that this ends up in dplyr one day -- this worked for the rows_*() family of functions in the past. Would that work for you?

@etiennebacher
Copy link

Yes, that seems to be the best approach, I could register the tidypolars methods if duckplyr is in the namespace. I'm just not sure how often both tidypolars and duckplyr are loaded together, while tidypolars requires loading dplyr so the integration would have been seamless if compute_*() functions were hosted in dplyr. But that's ok, maybe in the future as you say. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move compute_parquet etc. to upstream dplyr and make them generic functions

3 participants