[PATCH] ui: optionally refuse objects not reachable from any reference
Konstantin Ryabitsev
konstantin at linuxfoundation.org
Tue Aug 25 18:28:11 UTC 2026
When several repositories share an object database -- through
objects/info/alternates, or any other arrangement that deduplicates
identical objects across forks -- every object becomes readable from
every one of those repositories. cgit will render a commit page for a
commit belonging to an unrelated fork just as readily as for its own, so
a single commit has as many valid URLs as there are repositories sharing
the database.
Crawlers find all of them. Every one of those URLs generates a full
commit page, diff included, and claims its own cache slot, so a site
with hundreds of forks pays for hundreds of copies of the same page and
evicts the pages visitors actually asked for.
Add enable-object-reachability-check, off by default and overridable per
repository, which answers such a request with "404 Not found" instead.
The reference set mirrors the one shown on the refs page -- branches and
tags always, remote branches only when enable-remote-branches is set --
so that an object is never refused here while remaining visible there.
An object that is not commit-ish is answered differently. Whether a blob
or a tree occurs somewhere in the history cannot be determined without
walking every object in the repository, which no request can afford, so
such an object is displayed only when a reference points straight at it.
Those are exactly the objects cgit links to by raw id from its refs and
tag pages, so its own links keep working, while a request naming a blob
belonging to some other repository is refused. Reaching a blob by path
through a commit is governed by that commit and is unaffected.
A revision that does not resolve at all is left to the page handler,
which reports it in its own words. Conflating the two would tell the
visitor that an object is unknown when in fact it exists and simply does
not belong here. The refusal names only the revision the visitor
supplied and never reads the object, so it discloses nothing about the
commit and no diff is ever generated.
Snapshots take their revision from the filename rather than from the
query string, so they are checked separately, at the point where the
revision becomes known.
Answering the question means walking history, which is bounded by
generation numbers only when the repository has a commit-graph. Without
one the walk falls back to commit dates and is orders of magnitude
slower, so the documentation is explicit that this should not be enabled
on a repository whose maintenance does not include commit-graph writes.
That cost is one reason to leave the option off by default, and not the
main one. Where no object database is shared there is nothing for the
check to refuse, so most instances would gain nothing from it. It is
also a behaviour change: an object that is present but unreferenced is
served today and would stop being served, and such objects arise
routinely, from a branch that was reset or force-pushed before gc ran,
or from a fetch that has not been given a ref. Refusing them is the
right answer only for a site that knows its object database is shared.
Assisted-by: LLM [analysis, codegen, tests]
Signed-off-by: Konstantin Ryabitsev <konstantin at linuxfoundation.org>
---
On git.kernel.org forks share a single object database, so a commit that
lives in one tree has as many valid URLs as there are forks of it --
hundreds, in the popular cases -- and crawlers find every one of them.
We have been working around this with a commit filter that shells out to
git-branch --contains and prints a notice, which does not really help:
by the time a filter runs, the diff has already been generated, returned
and cached. Doing the check before dispatch is both cheaper and actually
effective.
Some numbers from a fork of the kernel tree, measuring the underlying
traversal:
reachable, recent commit 0.017s
reachable, old commit, with commit-graph 0.052s
reachable, old commit, without commit-graph 18.268s
unreachable commit 0.005s
The case crawlers actually hit, and the only one that leads to a
refusal, is the cheapest of the four: generation numbers reject it
without a walk. The third row is the one the documentation warns about.
Two choices that are not visible in the diff:
- The reachability half fails open. An empty repository, or an error out
of the traversal, counts as reachable, on the grounds that refusing to
serve a real page is the more damaging way to be wrong.
- Refusals are cached like any other id= page, which is to say with
cache-static-ttl and therefore, by default, forever. A refusal
outliving the condition that produced it is not ideal, but it is what
cgit already does for a 404 on a genuinely missing object, and giving
error responses their own lifetime is a wider change. I intend to
propose that separately.
And one question. A refused blob is reported with the same "not
reachable from any reference" message as a refused commit, which is a
stretch: such a blob may well be reachable in git's sense and merely
cannot be shown to be. Splitting the message is one more format string
if you would rather have it split.
---
cgit.c | 10 +++
cgit.h | 5 ++
cgitrc.5.txt | 31 ++++++++
shared.c | 111 +++++++++++++++++++++++++++
tests/t0112-object-reachability.sh | 150 +++++++++++++++++++++++++++++++++++++
ui-shared.c | 26 +++++++
ui-shared.h | 1 +
ui-snapshot.c | 9 +++
8 files changed, 343 insertions(+)
diff --git a/cgit.c b/cgit.c
index ca318e8..ce98a43 100644
--- a/cgit.c
+++ b/cgit.c
@@ -72,6 +72,8 @@ void cgit_repo_config(struct cgit_repo *repo, const char *name, const char *valu
repo->enable_log_filecount = atoi(value);
else if (!strcmp(name, "enable-log-linecount"))
repo->enable_log_linecount = atoi(value);
+ else if (!strcmp(name, "enable-object-reachability-check"))
+ repo->enable_object_reachability_check = atoi(value);
else if (!strcmp(name, "enable-remote-branches"))
repo->enable_remote_branches = atoi(value);
else if (!strcmp(name, "enable-subject-links"))
@@ -191,6 +193,8 @@ static void config_cb(const char *name, const char *value)
ctx.cfg.enable_log_filecount = atoi(value);
else if (!strcmp(name, "enable-log-linecount"))
ctx.cfg.enable_log_linecount = atoi(value);
+ else if (!strcmp(name, "enable-object-reachability-check"))
+ ctx.cfg.enable_object_reachability_check = atoi(value);
else if (!strcmp(name, "enable-remote-branches"))
ctx.cfg.enable_remote_branches = atoi(value);
else if (!strcmp(name, "enable-subject-links"))
@@ -762,6 +766,10 @@ static void process_request(void)
if (ctx.repo && prepare_repo_cmd(nongit))
return;
+ if (cgit_reject_unreachable_object(ctx.qry.oid) ||
+ cgit_reject_unreachable_object(ctx.qry.oid2))
+ return;
+
cmd->fn();
}
@@ -848,6 +856,8 @@ static void print_repo(FILE *f, struct cgit_repo *repo)
fprintf(f, "repo.logo=%s\n", repo->logo);
if (repo->logo_link)
fprintf(f, "repo.logo-link=%s\n", repo->logo_link);
+ fprintf(f, "repo.enable-object-reachability-check=%d\n",
+ repo->enable_object_reachability_check);
fprintf(f, "repo.enable-remote-branches=%d\n", repo->enable_remote_branches);
fprintf(f, "repo.enable-subject-links=%d\n", repo->enable_subject_links);
fprintf(f, "repo.enable-html-serving=%d\n", repo->enable_html_serving);
diff --git a/cgit.h b/cgit.h
index 7d7ece7..c7a62ce 100644
--- a/cgit.h
+++ b/cgit.h
@@ -7,6 +7,7 @@
#include <archive.h>
#include <commit.h>
+#include <commit-reach.h>
#include <diffcore.h>
#include <diff.h>
#include <environment.h>
@@ -104,6 +105,7 @@ struct cgit_repo {
int enable_follow_links;
int enable_log_filecount;
int enable_log_linecount;
+ int enable_object_reachability_check;
int enable_remote_branches;
int enable_subject_links;
int enable_html_serving;
@@ -238,6 +240,7 @@ struct cgit_config {
int enable_commit_graph;
int enable_log_filecount;
int enable_log_linecount;
+ int enable_object_reachability_check;
int enable_remote_branches;
int enable_subject_links;
int enable_html_serving;
@@ -347,6 +350,8 @@ extern void cgit_add_ref(struct reflist *list, struct refinfo *ref);
extern void cgit_free_reflist_inner(struct reflist *list);
extern int cgit_refs_cb(const struct reference *ref, void *cb_data);
+extern int cgit_oid_is_reachable(const struct object_id *oid);
+
extern void cgit_free_commitinfo(struct commitinfo *info);
extern void cgit_free_taginfo(struct taginfo *info);
diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 7c39bf9..4111eb3 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -202,6 +202,33 @@ enable-log-linecount::
and removed lines for each commit on the repository log page. Default
value: "0".
+enable-object-reachability-check::
+ Flag which, when set to "1", will make cgit refuse to display an
+ object that is not reachable from any reference in the repository,
+ answering with "404 Not found" instead. This is only useful where
+ several repositories share an object database, for example through
+ `objects/info/alternates`: every object is then readable from every
+ repository, so each object has as many valid URLs as there are
+ repositories sharing the database, and cgit will render and cache a
+ page for each one of them.
++
+The reference set used is the one shown on the refs page, so branches and
+tags always, and remote branches only when "enable-remote-branches" is
+set.
++
+Blobs and trees are treated differently, because whether one occurs
+somewhere in the history cannot be determined without walking every object
+in the repository. Such an object is displayed only when a reference points
+straight at it, which is the case cgit itself links to by raw object id.
+Requesting a blob by path through a commit is unaffected.
++
+Answering the question means walking history, which is affordable only
+when the repository has a commit-graph; without one the walk is bounded by
+commit dates rather than by generation numbers and takes orders of
+magnitude longer. Do not enable this on repositories of any size unless
+`git commit-graph write` is part of their maintenance. Default value: "0".
+See also: "repo.enable-object-reachability-check".
+
enable-remote-branches::
Flag which, when set to "1", will make cgit display remote branches
in the summary and refs views. Default value: "0". See also:
@@ -519,6 +546,10 @@ repo.enable-log-linecount::
A flag which can be used to disable the global setting
`enable-log-linecount'. Default value: none.
+repo.enable-object-reachability-check::
+ A flag which can be used to override the global setting
+ `enable-object-reachability-check'. Default value: none.
+
repo.enable-remote-branches::
Flag which, when set to "1", will make cgit display remote branches
in the summary and refs views. Default value: <enable-remote-branches>.
diff --git a/shared.c b/shared.c
index a39394d..6d1891b 100644
--- a/shared.c
+++ b/shared.c
@@ -66,6 +66,7 @@ struct cgit_repo *cgit_add_repo(const char *url)
ret->enable_follow_links = ctx.cfg.enable_follow_links;
ret->enable_log_filecount = ctx.cfg.enable_log_filecount;
ret->enable_log_linecount = ctx.cfg.enable_log_linecount;
+ ret->enable_object_reachability_check = ctx.cfg.enable_object_reachability_check;
ret->enable_remote_branches = ctx.cfg.enable_remote_branches;
ret->enable_subject_links = ctx.cfg.enable_subject_links;
ret->enable_html_serving = ctx.cfg.enable_html_serving;
@@ -224,6 +225,116 @@ int cgit_refs_cb(const struct reference *ref, void *cb_data)
return 0;
}
+struct reachability_tips {
+ struct commit **commits;
+ size_t nr;
+ size_t alloc;
+};
+
+static int collect_reachability_tip(const struct reference *ref, void *cb_data)
+{
+ struct reachability_tips *tips = cb_data;
+ struct commit *commit;
+
+ commit = lookup_commit_reference_gently(the_repository, ref->oid, 1);
+ if (!commit)
+ return 0;
+ ALLOC_GROW(tips->commits, tips->nr + 1, tips->alloc);
+ tips->commits[tips->nr++] = commit;
+ return 0;
+}
+
+/* Walk the references this repository publishes. The set deliberately mirrors
+ * the one shown by the refs page, so that an object is never refused here
+ * while remaining visible there, or the other way around.
+ */
+static void for_each_published_ref(refs_for_each_cb fn, void *cb_data)
+{
+ struct ref_store *refs = get_main_ref_store(the_repository);
+
+ if (refs_for_each_branch_ref(refs, fn, cb_data))
+ return;
+ if (refs_for_each_tag_ref(refs, fn, cb_data))
+ return;
+ if (ctx.repo && ctx.repo->enable_remote_branches)
+ refs_for_each_remote_ref(refs, fn, cb_data);
+}
+
+struct ref_target_match {
+ const struct object_id *oid;
+ int found;
+};
+
+static int match_ref_target(const struct reference *ref, void *cb_data)
+{
+ struct ref_target_match *match = cb_data;
+ struct object *obj;
+
+ if (oideq(ref->oid, match->oid)) {
+ match->found = 1;
+ return 1;
+ }
+ if (odb_read_object_info(the_repository->objects, ref->oid, NULL) != OBJ_TAG)
+ return 0;
+ obj = parse_object(the_repository, ref->oid);
+ if (!obj)
+ return 0;
+ obj = deref_tag(the_repository, obj, NULL, 0);
+ if (obj && oideq(&obj->oid, match->oid)) {
+ match->found = 1;
+ return 1;
+ }
+ return 0;
+}
+
+/* Determine whether 'oid' is published by this repository, meaning that a
+ * reference points directly at it or that it is reachable from one.
+ *
+ * For an object that is not commit-ish, only the first test is available.
+ * Asking whether a blob or a tree occurs somewhere in the history would mean
+ * walking every object in the repository, which no request can afford. The
+ * objects cgit links to by raw id are exactly the ones a reference points at,
+ * so the affordable test is also the one that matters: it keeps those links
+ * working and refuses everything else, which in a shared object database is
+ * every blob and tree belonging to some other repository.
+ *
+ * The reachability half reports the object as published when it cannot answer
+ * -- an empty repository has no tips to walk from, and an error out of the
+ * traversal cannot be interpreted -- because refusing to serve a page is the
+ * more damaging way to be wrong.
+ *
+ * Note that this half walks history, and is only affordable when the
+ * repository has a commit-graph; without one, the generation numbers that
+ * bound the walk are unavailable and the cost grows with the size of history.
+ */
+int cgit_oid_is_reachable(const struct object_id *oid)
+{
+ struct reachability_tips tips = { NULL, 0, 0 };
+ struct ref_target_match match = { oid, 0 };
+ struct commit *commit;
+ int reachable = 1;
+
+ commit = lookup_commit_reference_gently(the_repository, oid, 1);
+ if (!commit) {
+ /* An object that is not in the database at all is a different
+ * question, and one the page handler answers in its own words.
+ */
+ if (!odb_has_object(the_repository->objects, oid, 0))
+ return 1;
+ for_each_published_ref(match_ref_target, &match);
+ return match.found;
+ }
+
+ for_each_published_ref(collect_reachability_tip, &tips);
+
+ if (tips.nr)
+ reachable = repo_in_merge_bases_many(the_repository, commit,
+ tips.nr, tips.commits,
+ 1) != 0;
+ free(tips.commits);
+ return reachable;
+}
+
void cgit_diff_tree_cb(struct diff_queue_struct *q,
struct diff_options *options, void *data)
{
diff --git a/tests/t0112-object-reachability.sh b/tests/t0112-object-reachability.sh
new file mode 100755
index 0000000..30927fd
--- /dev/null
+++ b/tests/t0112-object-reachability.sh
@@ -0,0 +1,150 @@
+#!/bin/sh
+
+test_description='Verify enable-object-reachability-check'
+. ./setup.sh
+
+# Build a commit that exists in the object database but is not reachable from
+# any reference, which is what a repository sharing its object database with
+# others looks like from the inside. The marker keeps each such commit
+# distinct: the test dates are fixed, so two commits built the same way would
+# otherwise be the same object.
+mkunreachable() {
+ (
+ cd "$1" &&
+ echo "PAYLOAD$2" >leak.txt &&
+ git add leak.txt &&
+ tree=$(git write-tree) &&
+ git commit-tree -p HEAD -m "SUBJECT$2" "$tree"
+ )
+}
+
+test_expect_success 'setup' '
+ unreachable=$(mkunreachable repos/bar ONE) &&
+ tagged=$(mkunreachable repos/bar TWO) &&
+ nograph=$(mkunreachable repos/foo THREE) &&
+ (cd repos/bar && git reset -q --hard HEAD) &&
+ (cd repos/foo && git reset -q --hard HEAD) &&
+ (cd repos/bar && git tag -a -m "orphan" orphan "$tagged") &&
+ reachable=$(cd repos/bar && git rev-parse HEAD) &&
+ test "$unreachable" != "$tagged" &&
+ unreachable_blob=$(cd repos/bar && git rev-parse "$unreachable:leak.txt") &&
+ lightweight_blob=$(cd repos/bar &&
+ echo LIGHTWEIGHT | git hash-object -w --stdin) &&
+ annotated_blob=$(cd repos/bar &&
+ echo ANNOTATED | git hash-object -w --stdin) &&
+ (cd repos/bar && git tag blobtag "$lightweight_blob") &&
+ (cd repos/bar && git tag -a -m "blob" annblobtag "$annotated_blob") &&
+ cat >>cgitrc <<-EOF
+ repo.url=strict
+ repo.path=$PWD/repos/bar/.git
+ repo.desc=reachability checked
+ repo.enable-object-reachability-check=1
+
+ repo.url=strict-nograph
+ repo.path=$PWD/repos/foo/.git
+ repo.desc=reachability checked without a commit-graph
+ repo.enable-object-reachability-check=1
+ EOF
+'
+
+test_expect_success 'unreachable commit is served when the check is disabled' '
+ cgit_url "bar/commit/&id=$unreachable" >output &&
+ grep -q "SUBJECTONE" output &&
+ grep -q "PAYLOADONE" output
+'
+
+test_expect_success 'unreachable commit is refused when the check is enabled' '
+ cgit_url "strict/commit/&id=$unreachable" >output &&
+ grep -q "Status: 404 Not found" output &&
+ grep -q "is not reachable from any reference in this repository" output
+'
+
+test_expect_success 'the refusal discloses nothing about the commit' '
+ ! grep -q "SUBJECTONE" output &&
+ ! grep -q "PAYLOADONE" output
+'
+
+test_expect_success 'reachable commit is still served' '
+ cgit_url "strict/commit/&id=$reachable" >output &&
+ ! grep -q "Status: 404" output &&
+ grep -q "$reachable" output
+'
+
+test_expect_success 'a commit reachable only from a tag is served' '
+ cgit_url "strict/commit/&id=$tagged" >output &&
+ ! grep -q "Status: 404" output &&
+ grep -q "SUBJECTTWO" output
+'
+
+# Whether a blob occurs anywhere in the history cannot be answered without
+# walking every object in the repository, so the check accepts a blob only when
+# a reference points straight at it. Those are exactly the blobs cgit itself
+# links to by raw id, from the refs and tag pages.
+test_expect_success 'a blob is served by raw id when the check is disabled' '
+ cgit_url "bar/blob/&id=$unreachable_blob" >output &&
+ grep -q "PAYLOADONE" output
+'
+
+test_expect_success 'a blob no reference points at is refused' '
+ cgit_url "strict/blob/&id=$unreachable_blob" >output &&
+ grep -q "Status: 404 Not found" output &&
+ ! grep -q "PAYLOADONE" output
+'
+
+test_expect_success 'a blob a lightweight tag points at is served' '
+ cgit_url "strict/blob/&id=$lightweight_blob" >output &&
+ ! grep -q "Status: 404" output &&
+ grep -q "LIGHTWEIGHT" output
+'
+
+test_expect_success 'a blob an annotated tag points at is served' '
+ cgit_url "strict/blob/&id=$annotated_blob" >output &&
+ ! grep -q "Status: 404" output &&
+ grep -q "ANNOTATED" output
+'
+
+test_expect_success 'ordinary views are unaffected' '
+ cgit_url "strict/refs" >output &&
+ ! grep -q "Status: 404" output &&
+ cgit_url "strict/plain/file-50&id=$reachable" >output &&
+ ! grep -q "Status: 404" output
+'
+
+test_expect_success 'an object that does not resolve is still reported as such' '
+ cgit_url "strict/commit/&id=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef" >output &&
+ grep -q "Bad commit reference" output &&
+ ! grep -q "is not reachable from any reference" output
+'
+
+test_expect_success 'an unreachable id2 is refused by the diff view' '
+ cgit_url "strict/diff/&id=$reachable&id2=$unreachable" >output &&
+ grep -q "Status: 404 Not found" output &&
+ grep -q "is not reachable from any reference in this repository" output &&
+ ! grep -q "PAYLOADONE" output
+'
+
+test_expect_success 'an unreachable snapshot is refused' '
+ cgit_url "strict/snapshot/$unreachable.tar.gz" >output &&
+ grep -q "Status: 404 Not found" output &&
+ grep -q "is not reachable from any reference in this repository" output
+'
+
+test_expect_success 'the check works without a commit-graph' '
+ cgit_url "strict-nograph/commit/&id=$nograph" >output &&
+ grep -q "Status: 404 Not found" output &&
+ grep -q "is not reachable from any reference in this repository" output
+'
+
+# Answering the reachability question walks history, so the refusal is worth
+# caching: regenerating it for every crawler request would cost more than the
+# slot it occupies.
+test_expect_success 'a refusal is cached and replayed' '
+ rm -f cache/* &&
+ cgit_url "strict/commit/&id=$unreachable" >output &&
+ grep -q "Status: 404 Not found" output &&
+ test -n "$(ls cache)" &&
+ cgit_url "strict/commit/&id=$unreachable" >output.cached &&
+ test_cmp output output.cached
+'
+
+test_done
diff --git a/ui-shared.c b/ui-shared.c
index df52a9b..16c0524 100644
--- a/ui-shared.c
+++ b/ui-shared.c
@@ -886,6 +886,32 @@ void cgit_print_docend(void)
html("</body>\n</html>\n");
}
+/* Reject a request for an object that this repository does not publish. This
+ * only matters where several repositories share an object database: an object
+ * is then readable from every one of them, and each has as many valid URLs for
+ * it as there are repositories.
+ *
+ * A revision that does not resolve at all is left to the page handler, which
+ * reports it in its own words. Conflating the two would tell the visitor that
+ * an object is unknown when in fact it exists and simply does not belong here.
+ */
+int cgit_reject_unreachable_object(const char *rev)
+{
+ struct object_id oid;
+
+ if (!ctx.repo || !ctx.repo->enable_object_reachability_check)
+ return 0;
+ if (!rev || repo_get_oid(the_repository, rev, &oid))
+ return 0;
+ if (cgit_oid_is_reachable(&oid))
+ return 0;
+
+ cgit_print_error_page(404, "Not found",
+ "Object %s is not reachable from any reference "
+ "in this repository", rev);
+ return 1;
+}
+
void cgit_print_error_page(int code, const char *msg, const char *fmt, ...)
{
va_list ap;
diff --git a/ui-shared.h b/ui-shared.h
index 2a3a7f5..c581e43 100644
--- a/ui-shared.h
+++ b/ui-shared.h
@@ -71,6 +71,7 @@ extern void cgit_print_http_headers(void);
extern void cgit_redirect(const char *url, bool permanent);
extern void cgit_print_docstart(void);
extern void cgit_print_docend(void);
+extern int cgit_reject_unreachable_object(const char *rev);
__attribute__((format (printf,3,4)))
extern void cgit_print_error_page(int code, const char *msg, const char *fmt, ...);
extern void cgit_vprint_error_page(int code, const char *msg, const char *fmt, va_list ap);
diff --git a/ui-snapshot.c b/ui-snapshot.c
index d157222..a5007c9 100644
--- a/ui-snapshot.c
+++ b/ui-snapshot.c
@@ -305,6 +305,15 @@ void cgit_print_snapshot(const char *head, const char *hex,
if (!hex)
hex = head;
+ /* The revision may have come from the filename rather than from the
+ * query string, so it has not been checked by the dispatcher yet.
+ */
+ if (cgit_reject_unreachable_object(hex)) {
+ free(prefix);
+ free(adj_filename);
+ return;
+ }
+
if (!prefix)
prefix = xstrdup(cgit_snapshot_prefix(ctx.repo));
---
base-commit: 044821677c774cd24f25f1818ea51d09cc64b006
change-id: 20260825-object-reachability-check-b1db7249b02f
Best regards,
--
Konstantin Ryabitsev <konstantin at linuxfoundation.org>
More information about the CGit
mailing list