Backport #29297 by @KN4CK3R
Fixes#29101
Related #29298
Discard all read data to prevent misinterpreting existing data. Some
discard calls were missing in error cases.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: yp05327 <576951401@qq.com>
Backport #29298
Fixes the reason why #29101 is hard to replicate.
Related #29297
Create a repo with a file with minimum size 4097 bytes (I use 10000) and
execute the following code:
```go
gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>)
assert.NoError(t, err)
commit, err := gitRepo.GetCommit(<sha>)
assert.NoError(t, err)
entry, err := commit.GetTreeEntryByPath(<file>)
assert.NoError(t, err)
b := entry.Blob()
// Create a reader
r, err := b.DataAsync()
assert.NoError(t, err)
defer r.Close()
// Create a second reader
r2, err := b.DataAsync()
assert.NoError(t, err) // Should be no error but is ErrNotExist
defer r2.Close()
```
The problem is the check in `CatFileBatch`:
79217ea63c/modules/git/repo_base_nogogit.go (L81-L87)
`Buffered() > 0` is used to check if there is a "operation" in progress
at the moment. This is a problem because we can't control the internal
buffer in the `bufio.Reader`. The code above demonstrates a sequence
which initiates an operation for which the code thinks there is no
active processing. The second call to `DataAsync()` therefore reuses the
existing instances instead of creating a new batch reader.
Backport #29155 with an extra change: tolerate the git 2.43.1 GIT_FLUSH
bug in Gitea 1.21.x, more details in the comment of repo_attribute.go
Manually tested with git 2.43.1 and an old git (2.39.2)
Backport #29054. Fix#28840
This backport is for 1.21 only and it is different from the change in
1.22: this backport still accept the legacy date format to avoid
breaking.
Backport #28848 by @brechtvl
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.
Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.
But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.
According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.
Resolves#19810, #21148
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Backport #28824 by @lunny
`checkInit` has been invoked in `InitSimple`. So it's unnecessary to
invoke it twice in `InitFull`.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Backport #28348 by @AdamMajer
nogogit GetBranchNames() lists branches sorted in reverse commit date
order. On the other hand the gogit implementation doesn't apply any
ordering resulting in unpredictable behaviour. In my case, the unit
tests requiring particular order fail
repo_branch_test.go:24:
Error Trace:
./gitea/modules/git/repo_branch_test.go:24
Error: elements differ
extra elements in list A:
([]interface {}) (len=1) {
(string) (len=6) "master"
}
extra elements in list B:
([]interface {}) (len=1) {
(string) (len=7) "branch1"
}
listA:
([]string) (len=2) {
(string) (len=6) "master",
(string) (len=7) "branch2"
}
listB:
([]string) (len=2) {
(string) (len=7) "branch1",
(string) (len=7) "branch2"
}
Test: TestRepository_GetBranches
To fix this, we sort branches based on their commit date in gogit
implementation.
Fixes: #28318
Co-authored-by: Adam Majer <amajer@suse.de>
Backport #28356 by @darrinsmart
The summary string ends up in the database, and (at least) MySQL &
PostgreSQL require valid UTF8 strings.
Fixes#28178
Co-authored-by: darrinsmart <darrin@djs.to>
Co-authored-by: Darrin Smart <darrin@filmlight.ltd.uk>
Backport #28306 by @KN4CK3R
Fixes#28280
Reads the `previous` info from the `git blame` output instead of
calculating it afterwards.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Closes#26329
This PR adds the ability to ignore revisions specified in the
`.git-blame-ignore-revs` file in the root of the repository.
The banner is displayed in this case. I intentionally did not add a UI
way to bypass the ignore file (same behaviour as Github) but you can add
`?bypass-blame-ignore=true` to the url manually.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
From the Go specification:
> "1. For a nil slice, the number of iterations is 0."
https://go.dev/ref/spec#For_range
Therefore, an additional nil check for before the loop is unnecessary.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Hi,
We'd like to add merge files files to GetCommitFileStatus fucntions so
API returns the list of all the files associated to a merged pull
request commit, like GitHub API does.
The list of affectedFiles for an API commit is fetched from toCommit()
function in routers/api/v1/repo/commits.go, and API was returning no
file in case of a pull request with no conflict, or just files
associated to the confict resolution, but NOT the full list of merged
files.
This would lead to situations where a CI polling a repo for changes
could miss some file changes due to API returning an empty / partial
list in case of such merged pull requests. (Hope this makes sense :) )
NOTE: I'd like to add a unittest in
integrations/api_repo_git_commits_test.go but failed to understand how
to add my own test bare repo so I can make a test on a merged pull
request commit to check for affectedFiles.
Is there a merged pull request in there that I could use maybe?
Could someone please direct me to the relevant ressources with
informations on how to do that please?
Thanks for your time,
Laurent.
---------
Co-authored-by: Thomas Desveaux <desveaux.thomas@gmail.com>
Close stdout correctly for "git blame", otherwise the failed "git blame"
would case the request hanging forever.
And "os.Stderr" should never (seldom) be used as git command's stderr
Fix#26064
Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.
Fixes#26270.
Co-Author: @wxiaoguang
Thanks @lunny for providing this solution
As
https://github.com/go-gitea/gitea/issues/26270#issuecomment-1661695151
said, at present we cannot get the names of changed files correctly when
the `OldCommitID` is `EmptySHA`. In this PR, the `GetCommitFilesChanged`
method is added and will be used to get the changed files by commit ID.
References:
- https://stackoverflow.com/a/424142
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The `FileBlame` function looks strange, it has `revision` as argument
but doesn't use it.
Since the function never be used, I think we could just remove it.
If anyone thinks it should be kept, please help fix `revision`.
Co-authored-by: Giteabot <teabot@gitea.io>
To record which command is slow, this PR adds a debug log for slow git
operations.
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: delvh <dev.lh@web.de>
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.
Fix#15367
Replaces #23070
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.
We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.
Some answer from ChatGPT as ref.
> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
>
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
>
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
>
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
>
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.
---------
Co-authored-by: delvh <dev.lh@web.de>
Fix#24896
If users set different languages by `linguist-language`, the `stats` map
could be: `java: 100, Java: 200`.
Language stats are stored as case-insensitive in database and there is a
unique key.
So, the different language names should be merged to one unique name:
`Java: 300`
Close#13454 , Close#23255, Close#14697 (and maybe more related
issues)
Many users have the requirement to customize the git config. This PR
introduces an easy way: put the options in Gitea's app.ini
`[git.config]`, then the config options will be applied to git config.
And it can support more flexible default config values, eg: now
`diff.algorithm=histogram` by default. According to:
https://stackoverflow.com/a/32367597/4754037 , `histogram diff` is
efficient and doesn't like to cause server-side problems.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Giteabot <teabot@gitea.io>
## ⚠️ Breaking
The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.
Although many legacy options still work, it's encouraged to upgrade to
the new options.
The SMTP logger is deleted because SMTP is not suitable to collect logs.
If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.
## Description
Close#12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)
Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)
There is a new document (with examples): `logging-config.en-us.md`
This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.
## The old problems
The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.
## The new design
See `logger.go` for documents.
## Screenshot
<details>
</details>
## TODO
* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
Due to #24409 , we can now specify '--not' when getting all commits from
a repo to exclude commits from a different branch.
When I wrote that PR, I forgot to also update the code that counts the
number of commits in the repo. So now, if the --not option is used, it
may return too many commits, which can indicate that another page of
data is available when it is not.
This PR passes --not to the commands that count the number of commits in
a repo
For my specific use case, I'd like to get all commits that are on one
branch but NOT on the other branch.
For instance, I'd like to get all the commits on `Branch1` that are not
also on `master` (I.e. all commits that were made after `Branch1` was
created).
This PR adds a `not` query param that gets passed down to the `git log`
command to allow the user to exclude items from `GetAllCommits`.
See [git
documentation](https://git-scm.com/docs/git-log#Documentation/git-log.txt---not)
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Close#7570
1. Clearly define the wiki path behaviors, see
`services/wiki/wiki_path.go` and tests
2. Keep compatibility with old contents
3. Allow to use dashes in titles, eg: "2000-01-02 Meeting record"
4. Add a "Pages" link in the dropdown, otherwise users can't go to the
Pages page easily.
5. Add a "View original git file" link in the Pages list, even if some
file names are broken, users still have a chance to edit or remove it,
without cloning the wiki repo to local.
6. Fix 500 error when the name contains prefix spaces.
This PR also introduces the ability to support sub-directories, but it
can't be done at the moment due to there are a lot of legacy wiki data,
which use "%2F" in file names.
Co-authored-by: Giteabot <teabot@gitea.io>
Remove the misbehaving function and call
Repository.GetFilesChangedBetween instead.
Fixes#23919
---
~~_TODO_ test this~~ `Repository.getFilesChanged` seems to be only used
by Gitea Actions, but a similar function already exists
**Update** I tested this change and the issue is gone.
Reference:
https://github.com/go-gitea/gitea/issues/22578#issuecomment-1444180053
Credits to @tdesveaux , thank you very much for catching the problem. If
you'd like to open a PR, feel free to replace this one.
Git reports fatal errors for ambiguous arguments:
```
fatal: ambiguous argument 'refs/a...refs/b': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
```
So the `--` separator is necessary in some cases.
Currently gitea shows no commit information for files starting with a
colon.
[I set up a minimal repro repository that reproduces this error once
it's migrated on gitea](https://github.com/kbolashev/colon-test)
<img width="1209" alt="image"
src="https://user-images.githubusercontent.com/111061261/219326625-0e6d3a86-8b58-4d67-bc24-8a78963f36b9.png">
This is happening because the filenames piped to the `git log` command
are written as is, and it doesn't work when you have a colon at the
start of the filename, and you need to escape it.
You can test it locally, if you do
```
mkdir repo
git init
touch :file
git add . && git commit -m "Add file with colon"
git log -- :file
```
git log returns nothing. However, if you do `git log -- "\:file"`, it
will show the commit with the file change.
This PR escapes the starting colons in paths in the `LogNameStatusRepo`
function, making gitea return commit info about the file with the bad
filename.
<img width="1209" alt="image"
src="https://user-images.githubusercontent.com/111061261/219328299-46451246-4006-45e3-89b1-c244635ded23.png">
This error shows up only with files starting with colon, anywhere else
in filename is ok. Dashes at the beginning also seem to be working.
I don't know gitea internals well enough to know where else this error
can pop up, so I'm keeping this PR small as suggested by your
contributor guide
This PR adds support for reflogs on all repositories. It does this by
adding a global configuration entry.
Implements #14865
---------
Signed-off-by: Philip Peterson <philip.c.peterson@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
