Compare commits

..
Author SHA1 Message Date
jared 28c500f47a ci: mirror main and tags to the GitHub mirror
R-CMD-check / check (pull_request) Successful in 3m23s
R-CMD-check / check (push) Successful in 4m2s
A plain non-force git push rather than Gitea's built-in push mirror. A
push mirror force-updates the refs it owns, so a Merge clicked on a GitHub
PR would be silently overwritten on the next sync -- the PR still reading
'Merged' while its commit became unreachable. A non-force push is rejected
instead, which turns that into a red CI run.

Secret is PAT_GH, not GITHUB_MIRROR_PAT: Gitea reserves the GITHUB_ and
GITEA_ prefixes for its own injected variables and refuses secrets using
them.
2026-08-08 20:20:56 -04:00
3 changed files with 45 additions and 51 deletions
+43
View File
@@ -0,0 +1,43 @@
# Mirror the canonical Gitea repo to the public GitHub mirror.
#
# Deliberately a plain `git push`, NOT Gitea's built-in push mirror. A push
# mirror force-updates the refs it owns: if anyone ever clicks Merge on a
# GitHub PR, the next sync silently overwrites main, the PR still displays
# "Merged", the commit becomes unreachable, and nothing anywhere says so.
# A non-force push is REJECTED as non-fast-forward the moment that happens,
# turning a silent data-loss trap into a red CI run in a place we already look.
#
# Do NOT add --force here, and do NOT add GitHub branch protection to the
# mirror: protection rules block the mirror's legitimate pushes too, breaking
# normal syncing to catch an abnormal case.
#
# PAT_GH is a GitHub personal access token (repo + workflow scope; workflow is
# required because this pushes .github/workflows/). It is stored as a Gitea
# Actions secret. The name cannot begin with GITHUB_ or GITEA_ -- Gitea
# reserves both prefixes for its own injected variables and rejects the secret.
name: Mirror to GitHub
on:
push:
branches: [main]
tags: ['v*']
jobs:
mirror:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Push main and tags to the GitHub mirror
env:
PAT_GH: ${{ secrets.PAT_GH }}
run: |
set -eu
if [ -z "${PAT_GH:-}" ]; then
echo "PAT_GH is unset -- add it under Settings > Actions > Secrets." >&2
exit 1
fi
git push "https://x-access-token:${PAT_GH}@github.com/civilytics/uscogdata.git" \
HEAD:refs/heads/main --tags
+2 -18
View File
@@ -122,25 +122,9 @@
#' fallback -- correct for the local temp corpora the direct-execution tests
#' build.
#' @noRd
#' `fixed = TRUE` is load-bearing, not a style choice.
#'
#' In regex mode, `gsub()` interprets backslashes in the REPLACEMENT string as
#' escape sequences and silently drops them. A Windows corpus path is full of
#' them, so `C:\Users\RUNNER\AppData\...` was substituted in as
#' `C:UsersRUNNERAppData...` and every DuckDB read failed with "No files found
#' that match the pattern". `fixed = TRUE` treats pattern and replacement as
#' literal text, which is what a filesystem path needs.
#'
#' This is why the package could not read a LOCAL corpus on Windows at all --
#' including the test fixture, hence the entire suite, and any `cog_mirror()`
#' copy. Remote https URLs were unaffected, having no backslashes, which is
#' part of why it stayed hidden: the bug predates the `{long_files}` token and
#' lived in the original `{url}` substitution, unnoticed because nothing ever
#' ran on Windows until the mirror's check matrix existed.
#' @noRd
.render_view_sql <- function(sql, url, manifest = list()) {
sql <- gsub("{long_files}", .long_files_sql(url, manifest), sql, fixed = TRUE)
gsub("{url}", url, sql, fixed = TRUE)
sql <- gsub("\\{long_files\\}", .long_files_sql(url, manifest), sql, fixed = FALSE)
gsub("\\{url\\}", url, sql, fixed = FALSE)
}
#' Register DuckDB views from inst/sql/ SQL files
-33
View File
@@ -62,36 +62,3 @@ test_that("registered `long` view reads through the enumerated list", {
expect_true(all(c(2011, 2012, 2019, 2020) %in% yrs))
})
})
test_that("a Windows-style corpus path survives token substitution", {
# gsub() in regex mode treats backslashes in the REPLACEMENT as escape
# sequences and silently drops them, so a Windows path went in as
# C:\Users\RUNNER\... and came out as C:UsersRUNNER..., after which every
# DuckDB read failed with "No files found that match the pattern".
#
# That made a LOCAL corpus unreadable on Windows -- the bundled fixture
# included, so the whole suite failed there -- while remote https URLs
# worked fine, having no backslashes. It went unnoticed for the life of the
# package because nothing ever ran on Windows.
#
# Reproducible on any platform: this is string handling, not a filesystem
# behaviour, so it does not need a Windows runner to catch.
win <- "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\Rtmp123/"
out <- uscogdata:::.render_view_sql(
"FROM read_parquet('{url}data/summary_categories.parquet')", win
)
expect_true(grepl("C:\\Users\\RUNNER~1\\AppData", out, fixed = TRUE))
expect_false(grepl("C:Users", out, fixed = TRUE))
# The same must hold through the {long_files} path, which embeds the url
# once per enumerated partition.
manifest <- list(files = list(long_partitions = list(
list(year = 2011L, path = "data/long/year=2011/part-0.parquet")
)))
out2 <- uscogdata:::.render_view_sql(
"FROM read_parquet({long_files}, hive_partitioning = true)", win, manifest
)
expect_true(grepl("C:\\Users\\RUNNER~1\\AppData", out2, fixed = TRUE))
expect_false(grepl("C:Users", out2, fixed = TRUE))
})