From frfilips at gmail.com  Sat May  7 12:49:23 2022
From: frfilips at gmail.com (Filips R)
Date: Sat, 7 May 2022 15:49:23 +0300
Subject: Truncated output when writing to a pipe using sendfile
Message-ID: <CALe-fqgZ2inTKF1jkwLM7a=Gm1jpUz-+s0-tas1HdgLXu5M-5A@mail.gmail.com>

Hello!

I've been investigating an issue on my Alpine linux server running
cgit 1.2.3-r2.
When I call the CGI program and redirect to a regular file (or just
output to tty), the output is correct, however if the standard output
is a pipe, the output is truncated at 65523 bytes.

Here is the end of the diff between strace outputs:

 open("/var/cache/cgit/a1000000", O_RDONLY|O_LARGEFILE) = 3
 fstat(3, {st_mode=S_IFREG|0600, st_size=157003, ...}) = 0
 read(3, "st/tree/st.c\0Content-Type: text/"..., 4096) = 4096
-sendfile(1, 3, [13] => [157003], 156990) = 156990
+sendfile(1, 3, [13] => [65536], 156990) = 65523
 close(3)                                = 0
 exit_group(0)                           = ?
 +++ exited with 0 +++

I have managed to work around the issue by wrapping cgit and
redirecting to a temporary file.

Looks like the sendfile call comes from cache.c:95 -
https://git.zx2c4.com/cgit/tree/cache.c?h=v1.2.3&id=55fa25adb097d2681607d8b0f51a0c393cc9af1a#n95

senfile(2) says that a successful call may write fewer bytes than
requested; the caller should be prepared to retry the call if there
were unsent bytes

From dakkar at thenautilus.net  Sat May  7 13:03:51 2022
From: dakkar at thenautilus.net (Gianni Ceccarelli)
Date: Sat, 7 May 2022 14:03:51 +0100
Subject: Truncated output when writing to a pipe using sendfile
In-Reply-To: <CALe-fqgZ2inTKF1jkwLM7a=Gm1jpUz-+s0-tas1HdgLXu5M-5A@mail.gmail.com>
References: <CALe-fqgZ2inTKF1jkwLM7a=Gm1jpUz-+s0-tas1HdgLXu5M-5A@mail.gmail.com>
Message-ID: <20220507140351.5206874a@nautilus>

On 2022-05-07 Filips R <frfilips at gmail.com> wrote:
> Looks like the sendfile call comes from cache.c:95 -
> https://git.zx2c4.com/cgit/tree/cache.c?h=v1.2.3&id=55fa25adb097d2681607d8b0f51a0c393cc9af1a#n95
> 
> senfile(2) says that a successful call may write fewer bytes than
> requested; the caller should be prepared to retry the call if there
> were unsent bytes

Looks like the same issue as
https://www.mail-archive.com/cgit at lists.zx2c4.com/msg03158.html

-- 
	Dakkar - <Mobilis in mobile>
	GPG public key fingerprint = A071 E618 DD2C 5901 9574
	                             6FE2 40EA 9883 7519 3F88
	                    key id = 0x75193F88


From hristo at venev.name  Sat May  7 17:06:59 2022
From: hristo at venev.name (Hristo Venev)
Date: Sat,  7 May 2022 20:06:59 +0300
Subject: Truncated output when writing to a pipe using sendfile
In-Reply-To: <20220507140351.5206874a@nautilus>
References: <20220507140351.5206874a@nautilus>
Message-ID: <20220507170700.612935-1-hristo@venev.name>

Oops, I forgot about this...

Sorry about the duplicate emails. I'd made an error in one of the
addresses.



From hristo at venev.name  Sat May  7 17:07:00 2022
From: hristo at venev.name (Hristo Venev)
Date: Sat,  7 May 2022 20:07:00 +0300
Subject: [PATCH RESEND v2] cache: Tolerate short writes in print_slot
In-Reply-To: <20220507170700.612935-1-hristo@venev.name>
References: <20220507140351.5206874a@nautilus>
 <20220507170700.612935-1-hristo@venev.name>
Message-ID: <20220507170700.612935-2-hristo@venev.name>

sendfile() can return after a short read/write, so we may need to call
it more than once. As suggested in the manual page, we fall back to
read/write if sendfile fails with EINVAL or ENOSYS.

On the read/write path, use write_in_full which deals with short writes.

Signed-off-by: Hristo Venev <hristo at venev.name>
---
 cache.c | 45 +++++++++++++++++++++++++--------------------
 1 file changed, 25 insertions(+), 20 deletions(-)

diff --git a/cache.c b/cache.c
index 55199e8..1c843ba 100644
--- a/cache.c
+++ b/cache.c
@@ -85,40 +85,45 @@ static int close_slot(struct cache_slot *slot)
 /* Print the content of the active cache slot (but skip the key). */
 static int print_slot(struct cache_slot *slot)
 {
+	off_t off;
 #ifdef HAVE_LINUX_SENDFILE
-	off_t start_off;
-	int ret;
+	off_t size;
+#endif
+
+	off = slot->keylen + 1;
 
-	start_off = slot->keylen + 1;
+#ifdef HAVE_LINUX_SENDFILE
+	size = slot->cache_st.st_size;
 
 	do {
-		ret = sendfile(STDOUT_FILENO, slot->cache_fd, &start_off,
-				slot->cache_st.st_size - start_off);
+		ssize_t ret;
+		ret = sendfile(STDOUT_FILENO, slot->cache_fd, &off, size - off);
 		if (ret < 0) {
 			if (errno == EAGAIN || errno == EINTR)
 				continue;
+			/* Fall back to read/write on EINVAL or ENOSYS */
+			if (errno == EINVAL || errno == ENOSYS)
+				break;
 			return errno;
 		}
-		return 0;
+		if (off == size)
+			return 0;
 	} while (1);
-#else
-	ssize_t i, j;
+#endif
 
-	i = lseek(slot->cache_fd, slot->keylen + 1, SEEK_SET);
-	if (i != slot->keylen + 1)
+	if (lseek(slot->cache_fd, off, SEEK_SET) != off)
 		return errno;
 
 	do {
-		i = j = xread(slot->cache_fd, slot->buf, sizeof(slot->buf));
-		if (i > 0)
-			j = xwrite(STDOUT_FILENO, slot->buf, i);
-	} while (i > 0 && j == i);
-
-	if (i < 0 || j != i)
-		return errno;
-	else
-		return 0;
-#endif
+		ssize_t ret;
+		ret = xread(slot->cache_fd, slot->buf, sizeof(slot->buf));
+		if (ret < 0)
+			return errno;
+		if (ret == 0)
+			return 0;
+		if (write_in_full(STDOUT_FILENO, slot->buf, ret) < 0)
+			return errno;
+	} while (1);
 }
 
 /* Check if the slot has expired */
-- 
2.35.3


From Jason at zx2c4.com  Sat May  7 21:28:30 2022
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Sat, 7 May 2022 23:28:30 +0200
Subject: [PATCH] cache: Tolerate short writes in print_slot
In-Reply-To: <50038a48ec126969331acfa1df13b4ae6bdd3814.camel@venev.name>
References: <20210910141841.2092532-1-hristo@venev.name>
 <YV6/dYmy+OsWuTMQ@keeping.me.uk>
 <50038a48ec126969331acfa1df13b4ae6bdd3814.camel@venev.name>
Message-ID: <CAHmME9ozBFTb9zeOAL+S35Kp4NZ-x68zdT_mGJuTKgsXg_qtoA@mail.gmail.com>

Hi Hristo,

On Thu, Oct 7, 2021 at 5:45 PM Hristo Venev <hristo at venev.name> wrote:
> I will fix this in v2.

I'm finally culling patches for cgit. Did you want to submit a v2 of this patch?

Jason

From hristo at venev.name  Sat May  7 21:32:25 2022
From: hristo at venev.name (Hristo Venev)
Date: Sun, 08 May 2022 00:32:25 +0300
Subject: [PATCH] cache: Tolerate short writes in print_slot
In-Reply-To: <CAHmME9ozBFTb9zeOAL+S35Kp4NZ-x68zdT_mGJuTKgsXg_qtoA@mail.gmail.com>
References: <20210910141841.2092532-1-hristo@venev.name>
 <YV6/dYmy+OsWuTMQ@keeping.me.uk>
 <50038a48ec126969331acfa1df13b4ae6bdd3814.camel@venev.name>
 <CAHmME9ozBFTb9zeOAL+S35Kp4NZ-x68zdT_mGJuTKgsXg_qtoA@mail.gmail.com>
Message-ID: <bda5feb8d80bb76ecc963db519ac2e53921ed431.camel@venev.name>

On Sat, 2022-05-07 at 23:28 +0200, Jason A. Donenfeld wrote:
> Hi Hristo,
> 
> On Thu, Oct 7, 2021 at 5:45 PM Hristo Venev <hristo at venev.name>
> wrote:
> > I will fix this in v2.
> 
> I'm finally culling patches for cgit. Did you want to submit a v2 of
> this patch?

I submitted it earlier today.

https://lists.zx2c4.com/pipermail/cgit/2022-May/004723.html

> 
> Jason

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20220508/e6ba923b/attachment.sig>

From Jason at zx2c4.com  Sat May  7 21:38:24 2022
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Sat, 7 May 2022 23:38:24 +0200
Subject: [PATCH] cache: Tolerate short writes in print_slot
In-Reply-To: <bda5feb8d80bb76ecc963db519ac2e53921ed431.camel@venev.name>
References: <20210910141841.2092532-1-hristo@venev.name>
 <YV6/dYmy+OsWuTMQ@keeping.me.uk>
 <50038a48ec126969331acfa1df13b4ae6bdd3814.camel@venev.name>
 <CAHmME9ozBFTb9zeOAL+S35Kp4NZ-x68zdT_mGJuTKgsXg_qtoA@mail.gmail.com>
 <bda5feb8d80bb76ecc963db519ac2e53921ed431.camel@venev.name>
Message-ID: <CAHmME9rZA75vN1k4Xx9ob5P3JYtrjp7cgh6MwAtzEkzKpk7GtA@mail.gmail.com>

On Sat, May 7, 2022 at 11:32 PM Hristo Venev <hristo at venev.name> wrote:
> I submitted it earlier today.
>
> https://lists.zx2c4.com/pipermail/cgit/2022-May/004723.html

Thanks. Went to spam because the DKIM signature failed.

Jason

From list at eworm.de  Mon May  9 07:30:09 2022
From: list at eworm.de (Christian Hesse)
Date: Mon,  9 May 2022 09:30:09 +0200
Subject: [PATCH 1/1] git: update to v2.36.1
Message-ID: <20220509073009.35229-1-list@eworm.de>

From: Christian Hesse <mail at eworm.de>

Update to git version v2.36.1, no additional changes required.

Signed-off-by: Christian Hesse <mail at eworm.de>
---
 Makefile | 2 +-
 git      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 2509d21..93a61fd 100644
--- a/Makefile
+++ b/Makefile
@@ -14,7 +14,7 @@ htmldir = $(docdir)
 pdfdir = $(docdir)
 mandir = $(prefix)/share/man
 SHA1_HEADER = <openssl/sha.h>
-GIT_VER = 2.36.0
+GIT_VER = 2.36.1
 GIT_URL = https://www.kernel.org/pub/software/scm/git/git-$(GIT_VER).tar.xz
 INSTALL = install
 COPYTREE = cp -r
diff --git a/git b/git
index 6cd33dc..e54793a 160000
--- a/git
+++ b/git
@@ -1 +1 @@
-Subproject commit 6cd33dceed60949e2dbc32e3f0f5e67c4c882e1e
+Subproject commit e54793a95afeea1e10de1e5ad7eab914e7416250
-- 
2.36.1


From alx.manpages at gmail.com  Mon May  9 08:25:30 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Mon, 9 May 2022 10:25:30 +0200
Subject: Hyperlink for symlinks
Message-ID: <5d7f0077-bb63-2994-7ab5-96330a798daa@gmail.com>

Hi,

When a file is a symlink, cgit shows the TARGET string.  Normally, those 
symlinks will resolve to something within the same repository, and 
therefore I think it would be great if that text was itself a hyperlink 
to the TARGET file within the same repo.

Does it sound good to you?

Cheers,

Alex

From nilix at nilfm.cc  Mon May  9 14:57:25 2022
From: nilix at nilfm.cc (Derek Stevens)
Date: Mon, 09 May 2022 08:57:25 -0600
Subject: [PATCH] add 'go-import' meta tag if in a repo
In-Reply-To: <dc347023-fb80-4c44-b75a-1fd2b05e463c@www.fastmail.com>
References: <20220109174549.qjdox4wuav3mxtci@ksatrya.siroonian.local>
 <dc347023-fb80-4c44-b75a-1fd2b05e463c@www.fastmail.com>
Message-ID: <29WX9UX8ZXQCA.3PBY77LL70O3H@ksatrya.nilfm.local>

I've been really busy with life lately so I don't mind if you take it on!

Cheers,
Derek

"Max Resnick" <max_resnick at fastmail.fm> wrote:
> Hi are you going to complete this or would you care if I completed it? 
> 
> Thanks 
> Max
> 
> On Sun, Jan 9, 2022, at 09:45, Derek Stevens wrote:
> > Ok, I'll see about adding a config option and corresponding 
> > documentation for this, as well as make sure that my indentation is 
> > consistent.
> >
> > Thanks!
> > Derek
> >
> > ------original message------
> > From: Jamie Couture <jamie.couture at gmail.com>
> > Date: Sun, Jan 09, 2022 at 09:59:04AM MST
> >
> >> On Sat, Jan 08, 2022 at 04:55:59PM -0700, Derek Stevens wrote:
> >> > Signed-off-by: Derek Stevens <nilix at nilfm.cc>
> >> > ---
> >> > ui-shared.c | 3 +++
> >> > 1 file changed, 3 insertions(+)
> >> > 
> >> > diff --git a/ui-shared.c b/ui-shared.c
> >> > index acd8ab5..0f57af6 100644
> >> > --- a/ui-shared.c
> >> > +++ b/ui-shared.c
> >> > @@ -785,6 +785,9 @@ void cgit_print_docstart(void)
> >> > 	html_txt(ctx.page.title);
> >> > 	html("</title>\n");
> >> > 	htmlf("<meta name='generator' content='cgit %s'/>\n", cgit_version);
> >> > +	if (ctx.repo)
> >> I recommend making this an optional feature people can opt-in and use.  
> >> Consider adding a configuration flag [1] and supporting documentation [2],
> >> allowing admins to turn a knob 'on' if they want to use this feature.
> >> 
> >> As John pointed out a minor nit: the leading whitespace should be tabs, 
> >> whereas these lines are mixed tab-and-space..
> >> 
> >> > +	  htmlf("<meta name='go-import' content='%s%s%s git https://%s%s%s'/>\n",
> >> > +	    host, cgit_rooturl(), ctx.repo->url, host, cgit_rooturl(), ctx.repo->url);
> >> > 	if (ctx.cfg.robots && *ctx.cfg.robots)
> >> > 		htmlf("<meta name='robots' content='%s'/>\n", ctx.cfg.robots);
> >> > 	html("<link rel='stylesheet' type='text/css' href='");
> >> > -- 
> >> > 2.34.1
> >> > 
> >> 
> >> --
> >> 
> >> [1] https://git.zx2c4.com/cgit/tree/cgit.c#n42
> >> [2] https://git.zx2c4.com/cgit/tree/cgitrc.5.txt
> >
> >
> >
> > ----- End forwarded message -----
> >
> > Attachments:
> > * signature.asc



From june at causal.agency  Fri May 13 20:12:46 2022
From: june at causal.agency (june)
Date: Fri, 13 May 2022 16:12:46 -0400
Subject: Hyperlink for symlinks
In-Reply-To: <5d7f0077-bb63-2994-7ab5-96330a798daa@gmail.com>
References: <5d7f0077-bb63-2994-7ab5-96330a798daa@gmail.com>
Message-ID: <E8481DA0-050E-4AF4-AFD0-D3334A8AFFA9@causal.agency>

> On May 9, 2022, at 04:25, Alejandro Colomar <alx.manpages at gmail.com> wrote:
> 
> Hi,
> 
> When a file is a symlink, cgit shows the TARGET string.  Normally, those symlinks will resolve to something within the same repository, and therefore I think it would be great if that text was itself a hyperlink to the TARGET file within the same repo.

I implemented this ages ago:
https://git.causal.agency/cgit-pink/commit/?id=41a17adafa443dc5a0d42d5d21fb02230b9005b7


From june at causal.agency  Tue May 17 22:09:33 2022
From: june at causal.agency (june)
Date: Tue, 17 May 2022 18:09:33 -0400
Subject: [PATCH 1/1] git: update to v2.36.0
In-Reply-To: <20220418201739.35439-1-list@eworm.de>
References: <20220418201739.35439-1-list@eworm.de>
Message-ID: <8A9B1037-A45D-4F5C-ABFF-E9F031E0935E@causal.agency>

This introduces a bad free in cgit_diff_tree. Here's a fix:
https://git.causal.agency/cgit-pink/commit/?id=cc167887f1ee6907103533187ff9679f01006a1f