1
1
mirror of https://github.com/go-gitea/gitea synced 2025-07-05 18:17:19 +00:00
Commit Graph

157 Commits

Author SHA1 Message Date
468d1919b5 Fix missed merge commit sha and time when migrating from codecommit (#34645) (#34650)
Backport #34645 by @lunny

Fix #34627

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-06-08 21:21:27 -07:00
e18eae7129 Fix migration pull request title too long (#34577) (#34604)
Backport #34577 by @lunny

Fix #34294

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-06-04 13:36:09 -04:00
8965c068e9 Fix possible nil description of pull request when migrating from CodeCommit (#34541) (#34550)
Backport #34541 by @lunny

Fix #34320

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-05-27 20:11:09 +00:00
cdac263bb8 Only git operations should update last changed of a repository (#34388) (#34427)
Backport #34388 by @lunny

Try to fix #32046

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-05-11 23:23:12 +02:00
a3a95a0b67 Upgrade go-github v61 -> v71 (#34385) (#34387)
Backport #34385 by @lunny

There will be a possible bug when migrating from Github
https://github.com/google/go-github/issues/3229
This PR upgrades go-github from v61 to v71 to resolve that problem.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-05-07 10:05:41 -07:00
a9343896f4 Option to delay conflict checking of old pull requests until page view (#27779)
`[repository.pull-request] DELAY_CHECK_FOR_INACTIVE_DAYS` is a new
setting to delay the mergeable check for pull requests that have been
inactive for the specified number of days.

This avoids potentially long delays for big repositories with many pull
requests. and reduces system load overall when there are many
repositories or pull requests.

When viewing the PR, checking will start immediately and the PR merge
box will automatically reload when complete. Accessing the PR through
the API will also start checking immediately.

The default value of `7` provides a balance between system load, and
keeping behavior similar to what it was before both for users and API
access. With `0` all conflict checking will be delayed, while `-1`
always checks immediately to restore the previous behavior.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-24 19:26:57 +00:00
f6474cf2e9 Fix bug when migrating repository (#34182)
This PR fixed a bug which is a regression from #31035
2025-04-14 04:48:03 +00:00
8a6df00c53 fix github migration error when using multiple tokens (#34144)
Git authorization was not taking into account multiple token feature,
leading to auth failures

Closes: https://github.com/go-gitea/gitea/issues/34141

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-13 23:20:14 +00:00
ee3c82f874 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
0fde8ecd55 Enable testifylint rules (#34075)
enable testifylint rules disabled in:
https://github.com/go-gitea/gitea/pull/34054
2025-03-31 01:53:48 -04:00
51d86adb6d Fix some migration and repo name problems (#33986)
1. Ignore empty inputs in `UnmarshalHandleDoubleEncode`
2. Ignore non-existing `stateEvent.User` in gitlab migration
3. Enable `release` and `wiki` units when they are selected in migration
4. Sanitize repo name for migration and new repo
2025-03-24 20:26:58 -07:00
5d65b9060b remove context from retry downloader (#33871)
follow up to https://github.com/go-gitea/gitea/pull/33399 as I
apparently missed this one.
2025-03-14 00:28:57 +00:00
ae3a18e01a Remove context from git struct (#33793)
Argument is moved from struct init in command run, which lets us remove
context from struct.
2025-03-04 11:56:11 -08:00
cc1fdc84ca Use test context in tests and new loop system in benchmarks (#33648)
Replace all contexts in tests with go1.24 t.Context()

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-20 09:57:40 +00:00
1ec8d80fa3 refactor: decouple context from migration structs (#33399)
Use context as much as possible.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-07 05:37:32 +00:00
2a02734f93 Refactor older tests to use testify (#33140)
Refactor checks to use assert/require
Use require.Eventually for waiting in elastic and meilisearch tests
Use require to exit early instead of assert
2025-01-09 09:21:47 +08:00
e177239529 Fix typo in gitea downloader test and add missing codebase in ToGitServiceType (#33146) 2025-01-08 18:34:53 +00:00
124079871b Convert github.com/xanzy/go-gitlab into gitlab.com/gitlab-org/api/client-go (#33126)
Fix #32985
2025-01-08 13:10:43 +08:00
8eecca3478 Remove aws go sdk package dependency (#33029)
This PR removed the dependency of `github.com/aws/aws-sdk-go/aws`

Patially fix for #33023
2024-12-30 06:30:28 +00:00
9bfa9f450d Refactor "string truncate" (#32984) 2024-12-26 11:56:03 +08:00
751fe8b714 Move some errors to their own sub packages (#32880) 2024-12-20 18:05:29 +00:00
33e8e82c4b Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
74664b08a0 Support migrating GitHub/GitLab PR draft status (#32242)
Resolve #32196
2024-10-13 22:58:13 +03:00
70b7df0e5e Support repo license (#24872)
Close #278
Close #24076

## Solutions:
- Use
[google/licenseclassifier](https://github.com/google/licenseclassifier/)
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[go-license-detector](https://github.com/go-enry/go-license-detector):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167
Test result between
[google/licensecheck](https://github.com/google/licensecheck) and
[google/licenseclassifier](https://github.com/google/licenseclassifier/):
https://github.com/go-gitea/gitea/pull/24872#issuecomment-1576092178
- Generate License Convert Name List to avoid import license templates
with same contents
Gitea automatically get latest license data from[
spdx/license-list-data](https://github.com/spdx/license-list-data).
But unfortunately, some license templates have same contents. #20915
[click here to see the
list](https://github.com/go-gitea/gitea/pull/24872#issuecomment-1584141684)
So we will generate a list of these license templates with same contents
and create a new file to save the result when using `make
generate-license`. (Need to decide the save path)
- Save License info into a new table `repo_license`
Can easily support searching repo by license in the future.

## Screen shot
Single License:

![image](https://github.com/go-gitea/gitea/assets/18380374/41260bd7-0b4c-4038-8592-508706cffa9f)

Multiple Licenses:

![image](https://github.com/go-gitea/gitea/assets/18380374/34ce2f73-7e18-446b-9b96-ecc4fb61bd70)

Triggers:
- [x] Push commit to default branch
- [x] Create repo
- [x] Mirror repo
- [x] When Default Branch is changed, licenses should be updated

Todo:
- [x] Save Licenses info in to DB when there's a change to license file
in the commit
- [x] DB Migration
- [x] A nominal test?
- [x] Select which library to
use(https://github.com/go-gitea/gitea/pull/24872#issuecomment-1560361167)
- [x] API Support
- [x] Add repo license table
- ~Select license in settings if there are several licenses(Not
recommended)~
- License board(later, not in this PR)

![image](https://github.com/go-gitea/gitea/assets/18380374/2c3c3bf8-bcc2-4c6d-8ce0-81d1a9733878)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.com>
2024-10-01 15:25:08 -04:00
125679f2e1 Support allowed hosts for migrations to work with proxy (#32025) 2024-09-11 05:47:00 +00:00
def1c9670b Support migration from AWS CodeCommit (#31981)
This PR adds support for migrating repos from [AWS
CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/welcome.html).

The access key ID and secret access key are required to get repository
information and pull requests. And [HTTPS Git
credentials](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html)
are required to clone the repository.

<img
src="https://github.com/user-attachments/assets/82ecb2d0-8d43-42b0-b5af-f5347a13b9d0"
width="680" />

The AWS CodeCommit icon is from [AWS Architecture
Icons](https://aws.amazon.com/architecture/icons/).

<img
src="https://github.com/user-attachments/assets/3c44d21f-d753-40f5-9eae-5d3589e0d50d"
width="320" />
2024-09-11 07:49:42 +08:00
816222243a Add lint-go-gopls (#30729)
Uses `gopls check <files>` as a linter. Tested locally and brings up 149
errors currently for me. I don't think I want to fix them in this PR,
but I would like at least to get this analysis running on CI.

List of errors:
```
modules/indexer/code/indexer.go:181:11: impossible condition: nil != nil
routers/private/hook_post_receive.go:120:15: tautological condition: nil == nil
services/auth/source/oauth2/providers.go:185:9: tautological condition: nil == nil
services/convert/issue.go:216:11: tautological condition: non-nil != nil
tests/integration/git_test.go:332:9: impossible condition: nil != nil
services/migrations/migrate.go:179:24-43: unused parameter: ctx
services/repository/transfer.go:288:48-69: unused parameter: doer
tests/integration/api_repo_tags_test.go:75:41-61: unused parameter: session
tests/integration/git_test.go:696:64-74: unused parameter: baseBranch
tests/integration/gpg_git_test.go:265:27-39: unused parameter: t
tests/integration/gpg_git_test.go:284:23-29: unused parameter: tmpDir
tests/integration/gpg_git_test.go:284:31-35: unused parameter: name
tests/integration/gpg_git_test.go:284:37-42: unused parameter: email
```
2024-06-05 09:22:38 +08:00
edbf74c418 Fix "force private" logic (#31012)
When creating a repo, the "FORCE_PRIVATE" config option should be
respected, `readonly` doesn't work for checkbox, so it should use
`disabled` attribute.
2024-05-20 00:56:45 +00:00
e80466f734 Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
b2013be910 Bump github.com/google/go-github to v61 (#30738) 2024-04-28 01:20:23 -04:00
74f0c84fa4 Enable more revive linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
7af074dbee Change the default maxPerPage for gitbucket (#30392)
This patch improves the migration from gitbucket to gitea.

The gitbucket uses it's own internal perPage value (= 25) for paging and
ignore per_page arguments in the requested URL. This cause gitea to
migrate only 25 issues and 25 PRs from gitbucket repository. This may
not happens on old gitbucket. But recent gitbucket 4.40 or 4.38.4 has
this problem.

This patch change to use this internally hardcoded perPage of gitbucket
as gitea's maxPerPage numer when migrating from gitbucket. There are
several perPage values in gitbucket like 25 for Isseus/PRs and 10 for
Releases. Some of those API doesn't support paging yet. It sounds
difficult to implement, but using the minimum number among them worked
out very well. So, I use 10 in this patch.

Brief descriptions of problems and this patch are also available in
https://github.com/go-gitea/gitea/issues/30316.

In addition, I'm not sure what kind of test cases are possible to write
here. It's a test for migration, so it requires testing gitbucket server
and gitea server, I guess. Please let me know if it is possible to write
such test cases here. Thanks!
2024-04-11 22:51:40 -04:00
274bc00ca2 Fix duplicate migrated milestones (#30102)
Fix #17567
2024-03-26 10:20:26 +00:00
a3f05d0d98 remove util.OptionalBool and related functions (#29513)
and migrate affected code

_last refactoring bits to replace **util.OptionalBool** with
**optional.Option[bool]**_
2024-03-02 16:42:31 +01:00
58ce1de994 Move migration functions to services layer (#29497) 2024-03-01 02:23:00 +00:00
17f170ee37 Include resource state events in Gitlab downloads (#29382)
Some specific events on Gitlab issues and merge requests are stored
separately from comments as "resource state events". With this change,
all relevant resource state events are downloaded during issue and merge
request migration, and converted to comments.

This PR also updates the template used to render comments to add support
for migrated comments of these types.

ref: https://docs.gitlab.com/ee/api/resource_state_events.html
2024-02-26 05:08:21 +01:00
6e5966597c Properly migrate target branch change GitLab comment (#29340)
GitLab generates "system notes" whenever an event happens within the
platform. Unlike Gitea, those events are stored and retrieved as text
comments with no semantic details. The only way to tell whether a
comment was generated in this manner is the `system` flag on the note
type.

This PR adds detection for a new specific kind of event: Changing the
target branch of a PR. When detected, it is downloaded using Gitea's
type for this event, and eventually uploaded into Gitea in the expected
format, i.e. with no text content in the comment.

This PR also updates the template used to render comments to add support
for migrated comments of this type.

ref:
11bd6dc826/app/services/system_notes/merge_requests_service.rb (L102)
2024-02-24 07:49:16 +00:00
b79c30435f Use the database object format name but not read from git repoisitory everytime and fix possible migration wrong objectformat when migrating a sha256 repository (#29294)
Now we can get object format name from git command line or from the
database repository table. Assume the column is right, we don't need to
read from git command line every time.

This also fixed a possible bug that the object format is wrong when
migrating a sha256 repository from external.

<img width="658" alt="image"
src="https://github.com/go-gitea/gitea/assets/81045/6e9a9dcf-13bf-4267-928b-6bf2c2560423">
2024-02-24 06:55:19 +00:00
a70c00b80b Properly migrate automatic merge GitLab comments (#27873)
GitLab generates "system notes" whenever an event happens within the
platform. Unlike Gitea, those events are stored and retrieved as text
comments with no semantic details. The only way to tell whether a
comment was generated in this manner is the `system` flag on the note
type.

This PR adds detection for two specific kinds of events: Scheduling and
un-scheduling of automatic merges on a PR. When detected, they are
downloaded using Gitea's type for these events, and eventually uploaded
into Gitea in the expected format, i.e. with no text content in the
comment.

This PR also updates the template used to render comments to add support
for migrated comments of these two types.

ref:
11bd6dc826/app/services/system_notes/merge_requests_service.rb (L6-L17)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-02-22 07:29:03 +00:00
5f82ead13c Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.

## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:

Firstly, there are temporary repositories which do not change:

```go
git.OpenRepository(ctx, diskPath)
```

Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:

```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```

Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.

## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-01-27 21:09:51 +01:00
1af45689f9 Only migrate the first 255 chars of a Github issue title (#28902)
Fixes #28846
2024-01-24 02:09:08 +00:00
70c4aad8e1 Move more functions to db.Find (#28419)
Following #28220

This PR move more functions to use `db.Find`.

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-01-15 02:19:25 +00:00
d68b9237bf Use known issue IID to generate new PR index number when migrating from GitLab (#28616)
Fix #13884
2023-12-26 17:57:25 +00:00
cd2ff6e83d Bump google/go-github to v57 (#28514) 2023-12-18 15:42:04 -06:00
408a484224 Adjust object format interface (#28469)
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
2023-12-17 11:56:08 +00:00
cbf923e87b Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
537fa69962 Second part of refactor db.Find (#28194)
Continue of #27798 and move more functions to `db.Find` and `db.Count`.
2023-12-11 16:56:48 +08:00
49b98e45bc Fix migration panic due to an empty review comment diff (#28334)
Fix #28328 
```
func (p *PullRequestComment) GetDiffHunk() string {
	if p == nil || p.DiffHunk == nil {
		return ""
	}
	return *p.DiffHunk
}
```
This function in the package `go-github` may return an empty diff. When
it's empty, the following code will panic because it access `ss[1]`

ec1feedbf5/services/migrations/gitea_uploader.go (L861-L867)

ec1feedbf5/modules/git/diff.go (L97-L101)
2023-12-05 08:29:43 +00:00
df1e7d0067 Use db.Find instead of writing methods for every object (#28084)
For those simple objects, it's unnecessary to write the find and count
methods again and again.
2023-11-24 03:49:41 +00:00
ae396ac7c0 Fix DownloadFunc when migrating releases (#27887)
We should not use `asset.ID` in DownloadFunc because DownloadFunc is a
closure.

1bf5527eac/services/migrations/gitea_downloader.go (L284-L295)

A similar bug when migrating from GitHub has been fixed in #14703. This
PR fixes the bug when migrating from Gitea and GitLab.
2023-11-03 15:56:53 +08:00