1
1
mirror of https://github.com/go-gitea/gitea synced 2024-11-15 06:34:25 +00:00
Commit Graph

4618 Commits

Author SHA1 Message Date
Jason Song
1e4be0945b
Introduce globallock as distributed locks (#31908)
To help #31813, but do not replace it, since this PR just introduces the
new module but misses some work:

- New option in settings. `#31813` has done it.
- Use the locks in business logic. `#31813` has done it.

So I think the most efficient way is to merge this PR first (if it's
acceptable) and then finish #31813.

## Design principles

### Use spinlock even in memory implementation

In actual use cases, users may cancel requests. `sync.Mutex` will block
the goroutine until the lock is acquired even if the request is
canceled. And the spinlock is more suitable for this scenario since it's
possible to give up the lock acquisition.

Although the spinlock consumes more CPU resources, I think it's
acceptable in most cases.

### Do not expose the mutex to callers

If we expose the mutex to callers, it's possible for callers to reuse
the mutex, which causes more complexity.

For example:
```go
lock := GetLocker(key)
lock.Lock()
// ...
// even if the lock is unlocked, we cannot GC the lock,
// since the caller may still use it again.
lock.Unlock()
lock.Lock()
// ...
lock.Unlock()

// callers have to GC the lock manually.
RemoveLocker(key)
```

That's why
https://github.com/go-gitea/gitea/pull/31813#discussion_r1721200549

In this PR, we only expose `ReleaseFunc` to callers. So callers just
need to call `ReleaseFunc` to release the lock, and do not need to care
about the lock's lifecycle.
```go
_, release, err := locker.Lock(ctx, key)
if err != nil {
    return err
}
// ...
release()

// if callers want to lock again, they have to re-acquire the lock.
_, release, err := locker.Lock(ctx, key)
// ...
```

In this way, it's also much easier for redis implementation to extend
the mutex automatically, so that callers do not need to care about the
lock's lifecycle. See also
https://github.com/go-gitea/gitea/pull/31813#discussion_r1722659743

### Use "release" instead of "unlock"

For "unlock", it has the meaning of "unlock an acquired lock". So it's
not acceptable to call "unlock" when failed to acquire the lock, or call
"unlock" multiple times. It causes more complexity for callers to decide
whether to call "unlock" or not.

So we use "release" instead of "unlock" to make it clear. Whether the
lock is acquired or not, callers can always call "release", and it's
also safe to call "release" multiple times.

But the code DO NOT expect callers to not call "release" after acquiring
the lock. If callers forget to call "release", it will cause resource
leak. That's why it's always safe to call "release" without extra
checks: to avoid callers to forget to call it.

### Acquired locks could be lost

Unlike `sync.Mutex` which will be locked forever once acquired until
calling `Unlock`, in the new module, the acquired lock could be lost.

For example, the caller has acquired the lock, and it holds the lock for
a long time since auto-extending is working for redis. However, it lost
the connection to the redis server, and it's impossible to extend the
lock anymore.

If the caller don't stop what it's doing, another instance which can
connect to the redis server could acquire the lock, and do the same
thing, which could cause data inconsistency.

So the caller should know what happened, the solution is to return a new
context which will be canceled if the lock is lost or released:

```go
ctx, release, err := locker.Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
DoSomething(ctx)

// the lock is lost now, then ctx has been canceled.

// Failed, since ctx has been canceled.
DoSomethingElse(ctx)
```

### Multiple ways to use the lock

1. Regular way

```go
ctx, release, err := Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
```

2. Early release

```go
ctx, release, err := Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
// release the lock earlier and reset the context back
ctx = release()
// continue to do something else
// ...
```

3. Functional way

```go
if err := LockAndDo(ctx, key, func(ctx context.Context) error {
    // ...
    return nil
}); err != nil {
    return err
}
```
2024-08-26 22:27:57 +08:00
Lunny Xiao
0299bb97f0
Use correct function name (#31887) 2024-08-21 21:05:48 +00:00
Lunny Xiao
c03baab678
Refactor the usage of batch catfile (#31754)
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.

This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
2024-08-20 17:04:57 +00:00
胖梁
36232b69db
Actions support workflow dispatch event (#28163)
fix #23668 

My plan:
* In the `actions.list` method, if workflow is selected and IsAdmin,
check whether the on event contains `workflow_dispatch`. If so, display
a `Run workflow` button to allow the user to manually trigger the run.
* Providing a form that allows users to select target brach or tag, and
these parameters can be configured in yaml
* Simple form validation, `required` input cannot be empty
* Add a route `/actions/run`, and an `actions.Run` method to handle
* Add `WorkflowDispatchPayload` struct to pass the Webhook event payload
to the runner when triggered, this payload carries the `inputs` values
and other fields, doc: [workflow_dispatch
payload](https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_dispatch)

Other PRs
* the `Workflow.WorkflowDispatchConfig()` method still return non-nil
when workflow_dispatch is not defined. I submitted a PR
https://gitea.com/gitea/act/pulls/85 to fix it. Still waiting for them
to process.

Behavior should be same with github, but may cause confusion. Here's a
quick reminder.
*
[Doc](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch)
Said: This event will `only` trigger a workflow run if the workflow file
is `on the default branch`.
* If the workflow yaml file only exists in a non-default branch, it
cannot be triggered. (It will not even show up in the workflow list)
* If the same workflow yaml file exists in each branch at the same time,
the version of the default branch is used. Even if `Use workflow from`
selects another branch


![image](https://github.com/go-gitea/gitea/assets/3114995/4bf596f3-426b-48e8-9b8f-0f6d18defd79)
```yaml
name: Docker Image CI

on:
  workflow_dispatch:
    inputs:
      logLevel:
        description: 'Log level'
        required: true
        default: 'warning'
        type: choice
        options:
        - info
        - warning
        - debug
      tags:
        description: 'Test scenario tags'
        required: false
        type: boolean
      boolean_default_true:
        description: 'Test scenario tags'
        required: true
        type: boolean
        default: true
      boolean_default_false:
        description: 'Test scenario tags'
        required: false
        type: boolean
        default: false
      environment:
        description: 'Environment to run tests against'
        type: environment
        required: true
        default: 'environment values'
      number_required_1:
        description: 'number '
        type: number
        required: true
        default: '100'
      number_required_2:
        description: 'number'
        type: number
        required: true
        default: '100'
      number_required_3:
        description: 'number'
        type: number
        required: true
        default: '100'
      number_1:
        description: 'number'
        type: number
        required: false
      number_2:
        description: 'number'
        type: number
        required: false
      number_3:
        description: 'number'
        type: number
        required: false

env:
  inputs_logLevel:              ${{ inputs.logLevel }}
  inputs_tags:                  ${{ inputs.tags }}
  inputs_boolean_default_true:  ${{ inputs.boolean_default_true }}
  inputs_boolean_default_false: ${{ inputs.boolean_default_false }}
  inputs_environment:           ${{ inputs.environment }}
  inputs_number_1:              ${{ inputs.number_1  }}
  inputs_number_2:              ${{ inputs.number_2  }}
  inputs_number_3:              ${{ inputs.number_3  }}
  inputs_number_required_1:     ${{ inputs.number_required_1  }}
  inputs_number_required_2:     ${{ inputs.number_required_2  }}
  inputs_number_required_3:     ${{ inputs.number_required_3  }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: ls -la
      - run: env | grep inputs
      - run: echo ${{ inputs.logLevel }}
      - run: echo ${{ inputs.boolean_default_false }}
```

![image](https://github.com/go-gitea/gitea/assets/3114995/a58a842d-a0ff-4618-bc6d-83a9596d07c8)

![image](https://github.com/go-gitea/gitea/assets/3114995/44a7cca5-7bd4-42a9-8723-91751a501c88)

---------

Co-authored-by: TKaxv_7S <954067342@qq.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2024-08-18 22:38:40 -04:00
Zettat123
2010fbe060
Fix raw wiki links (#31825)
Fix #31395

This regression is introduced by #30273. To find out how GitHub handles
this case, I did [some
tests](https://github.com/go-gitea/gitea/issues/31395#issuecomment-2278929115).

I use redirect in this PR instead of checking if the corresponding `.md`
file exists when rendering the link because GitHub also uses redirect.
With this PR, there is no need to resolve the raw wiki link when
rendering a wiki page. If a wiki link points to a raw file, access will
be redirected to the raw link.
2024-08-16 20:40:51 +08:00
Zettat123
8883d99184
Support issue template assignees (#31083)
Resolve #13955
2024-08-12 16:00:40 +08:00
Jason Song
f4d3120f9d
Fix IsObjectExist with gogit (#31790)
Fix #31271.

When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.

So it causes #31271 because it reports that all blob objects do not
exist.
2024-08-09 02:40:45 +00:00
Jason Song
33cc5837a6
Support compression for Actions logs (#31761)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015

The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.

That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.

This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.

And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.

`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.

<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
2024-08-09 10:10:30 +08:00
Jason Song
de2787a493
Add TAGS to TEST_TAGS and fix bugs found with gogit (#31791)
Found at
https://github.com/go-gitea/gitea/pull/31790#issuecomment-2272898915

`unit-tests-gogit` never work since the workflow set `TAGS` with
`gogit`, but the Makefile use `TEST_TAGS`.

This PR adds the values of `TAGS` to `TEST_TAGS`, ensuring that setting
`TAGS` is always acceptable and avoiding confusion about which one
should be set.
2024-08-07 11:29:08 -04:00
Lunny Xiao
df7f1c2ead
Fix protected branch files detection on pre_receive hook (#31778)
Fix #31738

When pushing a new branch, the old commit is zero. Most git commands
cannot recognize the zero commit id. To get the changed files in the
push, we need to get the first diverge commit of this branch. In most
situations, we could check commits one by one until one commit is
contained by another branch. Then we will think that commit is the
diverge point.

And in a pre-receive hook, this will be more difficult because all
commits haven't been merged and they actually stored in a temporary
place by git. So we need to bring some envs to let git know the commit
exist.
2024-08-06 13:32:49 +00:00
Exploding Dragon
de175e3b06
Add signature support for the RPM module (#27069)
close  #27031

If the rpm package does not contain a matching gpg signature, the
installation will fail. See (#27031) , now auto-signing rpm uploads.

This option is turned off by default for compatibility.
2024-08-06 09:03:33 -04:00
Edip Emre Bodur
94cca8846e
Fix null requested_reviewer from API (#31773)
If the assign the pull request review to a team, it did not show the
members of the team in the "requested_reviewers" field, so the field was
null. As a solution, I added the team members to the array.

fix #31764
2024-08-05 10:59:53 +00:00
Jason Song
687c118248
Clear up old Actions logs (#31735)
Part of #24256.

Clear up old action logs to free up storage space.

Users will see a message indicating that the log has been cleared if
they view old tasks.

<img width="1361" alt="image"
src="https://github.com/user-attachments/assets/9f0f3a3a-bc5a-402f-90ca-49282d196c22">

Docs: https://gitea.com/gitea/docs/pulls/40

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-08-02 00:42:08 +00:00
Jason Song
333c9ed8ca
Add permission description for API to add repo collaborator (#31744)
Fix #31552.
2024-08-01 09:33:40 +00:00
Jason Song
09b56fc069
Distinguish LFS object errors to ignore missing objects during migration (#31702)
Fix #31137.

Replace #31623 #31697.

When migrating LFS objects, if there's any object that failed (like some
objects are losted, which is not really critical), Gitea will stop
migrating LFS immediately but treat the migration as successful.

This PR checks the error according to the [LFS api
doc](https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md#successful-responses).

> LFS object error codes should match HTTP status codes where possible:
> 
> - 404 - The object does not exist on the server.
> - 409 - The specified hash algorithm disagrees with the server's
acceptable options.
> - 410 - The object was removed by the owner.
> - 422 - Validation error.

If the error is `404`, it's safe to ignore it and continue migration.
Otherwise, stop the migration and mark it as failed to ensure data
integrity of LFS objects.

And maybe we should also ignore others errors (maybe `410`? I'm not sure
what's the difference between "does not exist" and "removed by the
owner".), we can add it later when some users report that they have
failed to migrate LFS because of an error which should be ignored.
2024-07-31 10:29:48 +00:00
Kemal Zebari
c0b5a843ba
Properly filter issue list given no assignees filter (#31522)
Quick fix #31520. This issue is related to #31337.
2024-07-23 18:36:32 +00:00
Shivaram Lingamneni
2f1cb1d289
fix OIDC introspection authentication (#31632)
See discussion on #31561 for some background.

The introspect endpoint was using the OIDC token itself for
authentication. This fixes it to use basic authentication with the
client ID and secret instead:

* Applications with a valid client ID and secret should be able to
  successfully introspect an invalid token, receiving a 200 response
  with JSON data that indicates the token is invalid
* Requests with an invalid client ID and secret should not be able
  to introspect, even if the token itself is valid

Unlike #31561 (which just future-proofed the current behavior against
future changes to `DISABLE_QUERY_AUTH_TOKEN`), this is a potential
compatibility break (some introspection requests without valid client
IDs that would previously succeed will now fail). Affected deployments
must begin sending a valid HTTP basic authentication header with their
introspection requests, with the username set to a valid client ID and
the password set to the corresponding client secret.
2024-07-23 12:43:03 +00:00
Denys Konovalov
a8d0c879c3
add skip secondary authorization option for public oauth2 clients (#31454) 2024-07-19 14:28:30 -04:00
Carsten Klein
3571b7e3dd
Allow searching issues by ID (#31479)
When you are entering a number in the issue search, you likely want the
issue with the given ID (code internal concept: issue index).
As such, when a number is detected, the issue with the corresponding ID
will now be added to the results.

Fixes #4479

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-07-17 00:49:05 +02:00
6543
0f53324182
Add option to change mail from user display name (#31528)
Make it posible to let mails show e.g.:

`Max Musternam (via gitea.kithara.com) <gitea@kithara.com>`

Docs: https://gitea.com/gitea/docs/pulls/23

---
*Sponsored by Kithara Software GmbH*
2024-07-14 23:27:00 +02:00
6543
1064e817c4
Issue Templates: add option to have dropdown printed list (#31577)
Issue template dropdown can have many entries, and it could be better to
have them rendered as list later on if multi-select is enabled.

so this adds an option to the issue template engine to do so.

DOCS: https://gitea.com/gitea/docs/pulls/19

---

## demo:

```yaml
name: Name
title: Title
about: About
labels: ["label1", "label2"]
ref: Ref
body:
  - type: dropdown
    id: id6
    attributes:
      label: Label of dropdown (list)
      description: Description of dropdown
      multiple: true
      list: true
      options:
        - Option 1 of dropdown
        - Option 2 of dropdown
        - Option 3 of dropdown
        - Option 4 of dropdown
        - Option 5 of dropdown
        - Option 6 of dropdown
        - Option 7 of dropdown
        - Option 8 of dropdown
        - Option 9 of dropdown
```


![image](https://github.com/user-attachments/assets/102ed0f4-89da-420b-ab2a-1788b59676f9)

![image](https://github.com/user-attachments/assets/a2bdb14e-43ff-4cc6-9bbe-20244830453c)


---
*Sponsored by Kithara Software GmbH*
2024-07-14 16:38:45 +02:00
Rowan Bohde
1ee59f0fa3
Allow disabling authentication related user features (#31535)
We have some instances that only allow using an external authentication
source for authentication. In this case, users changing their email,
password, or linked OpenID connections will not have any effect, and
we'd like to prevent showing that to them to prevent confusion.

Included in this are several changes to support this:
* A new setting to disable user managed authentication credentials
(email, password & OpenID connections)
* A new setting to disable user managed MFA (2FA codes & WebAuthn)
* Fix an issue where some templates had separate logic for determining
if a feature was disabled since it didn't check the globally disabled
features
* Hide more user setting pages in the navbar when their settings aren't
enabled

---------

Co-authored-by: Kyle D <kdumontnu@gmail.com>
2024-07-09 17:36:31 +00:00
Henry Goodman
12cb1d2998
Allow force push to protected branches (#28086)
Fixes #22722 

### Problem
Currently, it is not possible to force push to a branch with branch
protection rules in place. There are often times where this is necessary
(CI workflows/administrative tasks etc).

The current workaround is to rename/remove the branch protection,
perform the force push, and then reinstate the protections.

### Solution
Provide an additional section in the branch protection rules to allow
users to specify which users with push access can also force push to the
branch. The default value of the rule will be set to `Disabled`, and the
UI is intuitive and very similar to the `Push` section.

It is worth noting in this implementation that allowing force push does
not override regular push access, and both will need to be enabled for a
user to force push.

This applies to manual force push to a remote, and also in Gitea UI
updating a PR by rebase (which requires force push)

This modifies the `BranchProtection` API structs to add:
- `enable_force_push bool`
- `enable_force_push_whitelist bool`
- `force_push_whitelist_usernames string[]`
- `force_push_whitelist_teams string[]`
- `force_push_whitelist_deploy_keys bool`

### Updated Branch Protection UI:

<img width="943" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/7491899c-d816-45d5-be84-8512abd156bf">

### Pull Request `Update branch by Rebase` option enabled with source
branch `test` being a protected branch:


![image](https://github.com/go-gitea/gitea/assets/79623665/e018e6e9-b7b2-4bd3-808e-4947d7da35cc)
<img width="1038" alt="image"
src="https://github.com/go-gitea/gitea/assets/79623665/57ead13e-9006-459f-b83c-7079e6f4c654">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-07-05 18:21:56 +00:00
Brecht Van Lommel
b88e5fc72d
Fix slow patch checking with commits that add or remove many files (#31548)
Running git update-index for every individual file is slow, so add and
remove everything with a single git command.

When such a big commit lands in the default branch, it could cause PR
creation and patch checking for all open PRs to be slow, or time out
entirely. For example, a commit that removes 1383 files was measured to
take more than 60 seconds and timed out. With this change checking took
about a second.

This is related to #27967, though this will not help with commits that
change many lines in few files.
2024-07-04 18:57:11 +00:00
charles
f0033051d5
Fix markdown preview $$ support (#31514)
close #31481

currently `$$A + B$$ test` will ignore text after $$ block

test text

![圖片](https://github.com/go-gitea/gitea/assets/30816317/39b2974b-c0b6-48a0-87d0-5f4a13615eed)

before fix

![圖片](https://github.com/go-gitea/gitea/assets/30816317/15469e4c-474d-4128-b46f-d6cadaafbd68)

after fix

![圖片](https://github.com/go-gitea/gitea/assets/30816317/c1025eef-177f-4ade-988f-510e7039f3f9)

github display

![圖片](https://github.com/go-gitea/gitea/assets/30816317/97cd1e10-ac94-4899-86d8-8e359ef1d694)
2024-06-29 23:23:47 +00:00
Anbraten
91745ae46f
Add Passkey login support (#31504)
closes #22015

After adding a passkey, you can now simply login with it directly by
clicking `Sign in with a passkey`.

![Screenshot from 2024-06-26
12-18-17](https://github.com/go-gitea/gitea/assets/6918444/079013c0-ed70-481c-8497-4427344bcdfc)

Note for testing. You need to run gitea using `https` to get the full
passkeys experience.

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-06-29 22:50:03 +00:00
Royce Remer
df805d6ed0
Support legacy _links LFS batch responses (#31513)
Support legacy _links LFS batch response.

Fixes #31512.

This is backwards-compatible change to the LFS client so that, upon
mirroring from an upstream which has a batch api, it can download
objects whether the responses contain the `_links` field or its
successor the `actions` field. When Gitea must fallback to the legacy
`_links` field a logline is emitted at INFO level which looks like this:
```
...s/lfs/http_client.go:188:performOperation() [I] <LFSPointer ee95d0a27ccdfc7c12516d4f80dcf144a5eaf10d0461d282a7206390635cdbee:160> is using a deprecated batch schema response!
```

I've only run `test-backend` with this code, but added a new test to
cover this case. Additionally I have a fork with this change deployed
which I've confirmed syncs LFS from Gitea<-Artifactory (which has legacy
`_links`) as well as from Gitea<-Gitea (which has the modern `actions`).

Signed-off-by: Royce Remer <royceremer@gmail.com>
2024-06-28 08:42:57 +00:00
techknowlogick
a4899ffa24
bump golang deps (#31422) 2024-06-24 13:26:42 +00:00
Sergey Sharybin
49b8716c40
Support relative paths to videos from Wiki pages (#31061)
This change fixes cases when a Wiki page refers to a video stored in the
Wiki repository using relative path. It follows the similar case which
has been already implemented for images.

Test plan:
- Create repository and Wiki page
- Clone the Wiki repository
- Add video to it, say `video.mp4`
- Modify the markdown file to refer to the video using `<video
src="video.mp4">`
- Commit the Wiki page
- Observe that the video is properly displayed

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-06-21 18:23:54 +00:00
charles
90a3c20e79
Fix markdown math brackets render problem (#31420)
Close #31371, support `($ ... $)` like GitHub

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-06-20 02:12:54 +00:00
Tobias Balle-Petersen
1c26127b52
Increase max length of org team names from 30 to 255 characters (#31410)
This PR modifies the structs for editing and creating org teams to allow
team names to be up to 255 characters. The previous maximum length was
30 characters.
2024-06-19 10:36:09 -04:00
wxiaoguang
43c7a2e7b1
Refactor names (#31405)
This PR only does "renaming":

* `Route` should be `Router` (and chi router is also called "router")
* `Params` should be `PathParam` (to distingush it from URL query param, and to match `FormString`)
* Use lower case for private functions to avoid exposing or abusing
2024-06-19 06:32:45 +08:00
wxiaoguang
21783a5752
Fix rendered wiki page link (#31398)
Fix #31395
2024-06-18 03:09:20 +00:00
wxiaoguang
d32648b204
Refactor route path normalization (#31381)
Refactor route path normalization and decouple it from the chi router.
Fix the TODO, fix the legacy strange path behavior.
2024-06-17 23:28:47 +00:00
wxiaoguang
5a7376c060
Refactor markup code (#31399)
1. use clearer names
2. remove deadcode
3. avoid name shadowing
4. eliminate some lint warnings
2024-06-17 22:56:45 +00:00
6543
363c123598
Add cache test for admins (#31265)
Add a test to probe the cache similar to the email test func.


![image](https://github.com/go-gitea/gitea/assets/24977596/700e2733-586d-4091-900f-f5f71e6e94bf)


![image](https://github.com/go-gitea/gitea/assets/24977596/2a953802-18fc-4e81-a37d-24ebe1297365)


![image](https://github.com/go-gitea/gitea/assets/24977596/e00d62ad-bb60-41cc-9138-09993daee156)

---------

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: silverwind <me@silverwind.io>
2024-06-17 21:22:39 +02:00
wxiaoguang
25f3ec5b65
Fix natural sort (#31384)
Fix #31374
2024-06-17 06:45:12 +00:00
Brecht Van Lommel
597d1da96b
Fix missing images in editor preview due to wrong links (#31299)
Parse base path and tree path so that media links can be correctly
created with /media/.

Resolves #31294

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-06-17 14:16:14 +08:00
wxiaoguang
84cbb6c4d2
Fix duplicate sub-path for avatars (#31365)
Fix #31361, and add tests

And this PR introduces an undocumented & debug-purpose-only config
option: `USE_SUB_URL_PATH`. It does nothing for end users, it only helps
the development of sub-path related problems.

And also fix #31366

Co-authored-by: @ExplodingDragon
2024-06-15 11:43:57 +08:00
mzroot
d4e4226c3c
Add tag protection via rest api #17862 (#31295)
Add tag protection manage via rest API.

---------

Co-authored-by: Alexander Kogay <kogay.a@citilink.ru>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-06-14 18:56:10 +02:00
KN4CK3R
4e7b067a7f
Extract and display readme and comments for Composer packages (#30927)
Related #30075

CC @thojo0

Example with rendered readme:

![grafik](https://github.com/go-gitea/gitea/assets/1666336/3516fef5-2631-40fd-8841-5d9894ec8904)
2024-06-14 04:45:52 +00:00
Lunny Xiao
e4abaff7ff
Fix bug filtering issues which have no project (#31337)
Fix #31327
This is a quick patch to fix the bug.
Some parameters are using 0, some are using -1. I think it needs a
refactor to keep consistent. But that will be another PR.
2024-06-14 02:31:07 +00:00
Oleksandr Redko
1761459ebc
Refactor to use UnsafeStringToBytes (#31358)
The PR replaces all `goldmark/util.BytesToReadOnlyString` with
`util.UnsafeBytesToString`, `goldmark/util.StringToReadOnlyBytes` with
`util.UnsafeStringToBytes`. This removes one `TODO`.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-06-14 01:26:33 +00:00
Lunny Xiao
7115dce773
Fix hash render end with colon (#31319)
Fix a hash render problem like `<hash>: xxxxx` which is usually used in
release notes.
2024-06-12 22:35:46 +00:00
Lunny Xiao
1968c2222d
Fix adopt repository has empty object name in database (#31333)
Fix #31330
Fix #31311

A workaround to fix the old database is to update object_format_name to
`sha1` if it's empty or null.
2024-06-12 18:22:01 +08:00
Zoupers Zou
e25d6960b5
Fix #31185 try fix lfs download from bitbucket failed (#31201)
Fix #31185
2024-06-11 22:22:28 +00:00
silverwind
fc2d75f86d
Enable unparam linter (#31277)
Enable [unparam](https://github.com/mvdan/unparam) linter.

Often I could not tell the intention why param is unused, so I put
`//nolint` for those cases like webhook request creation functions never
using `ctx`.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2024-06-11 18:47:45 +00:00
wxiaoguang
4bf848a06b
Make template Iif exactly match if (#31322) 2024-06-11 14:52:12 +00:00
Kerwin Bryant
e6ab6e637f
code optimization (#31315)
Simplifying complex if-else to existing Iif operations
2024-06-11 21:07:10 +08:00
wxiaoguang
5342a61124
Delete legacy cookie before setting new cookie (#31306)
Try to fix #31202
2024-06-11 11:31:23 +08:00
wxiaoguang
6106a61eff
Remove sub-path from container registry realm (#31293)
Container registry requires that the "/v2" must be in the root, so the
sub-path in AppURL should be removed
2024-06-09 16:29:29 +08:00
Henrique Pimentel
f7125ab61a
Add MAX_ROWS option for CSV rendering (#30268)
This solution implements a new config variable MAX_ROWS, which
corresponds to the “Maximum allowed rows to render CSV files. (0 for no
limit)” and rewrites the Render function for CSV files in markup module.
Now the render function only reads the file once, having MAX_FILE_SIZE+1
as a reader limit and MAX_ROWS as a row limit. When the file is larger
than MAX_FILE_SIZE or has more rows than MAX_ROWS, it only renders until
the limit, and displays a user-friendly warning informing that the
rendered data is not complete, in the user's language.

---

Previously, when a CSV file was larger than the limit, the render
function lost its function to render the code. There were also multiple
reads to the file, in order to determine its size and render or
pre-render.

The warning: ![image](https://s3.amazonaws.com/i.snag.gy/vcKh90.jpg)
2024-06-06 16:06:59 +08:00
wxiaoguang
bd80225ec3
Make blockquote attention recognize more syntaxes (#31240)
Fix #31214
2024-06-04 15:35:29 +00:00
wxiaoguang
9000811118
Make pasted "img" tag has the same behavior as markdown image (#31235)
Fix #31230

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2024-06-04 12:19:41 +00:00
6543
aace3bccc3
Add option for mailer to override mail headers (#27860)
Add option to override headers of mails, gitea send out

---
*Sponsored by Kithara Software GmbH*
2024-06-03 20:42:52 +02:00
Kemal Zebari
cb27c438a8
Document possible action types for the user activity feed API (#31196)
Resolves #31131.

It uses the the go-swagger `enum` property to document the activity
action types.
2024-06-03 13:40:48 +00:00
wxiaoguang
a4275951ba
Split sanitizer functions and fine-tune some tests (#31192) 2024-05-31 13:26:01 +00:00
Lunny Xiao
fb7b743bd0
Azure blob storage support (#30995)
This PR implemented object storages(LFS/Packages/Attachments and etc.)
for Azure Blob Storage. It depends on azure official golang SDK and can
support both the azure blob storage cloud service and azurite mock
server.

Replace #25458
Fix #22527 

- [x] CI Tests
- [x] integration test, MSSQL integration tests will now based on
azureblob
  - [x] unit test 
- [x] CLI Migrate Storage
- [x] Documentation for configuration added

------

TODO (other PRs):
- [ ] Improve performance of `blob download`.

---------

Co-authored-by: yp05327 <576951401@qq.com>
2024-05-30 07:33:50 +00:00
Lunny Xiao
015efcd8bf
Use repo as of renderctx's member rather than a repoPath on metas (#29222)
Use a `gitrepo.Repository` in the markup's RenderContext but not store
the repository's path.
2024-05-30 07:04:01 +00:00
Lunny Xiao
de4616690f
Add topics for repository API (#31127)
Fix ##31100
2024-05-28 10:03:54 +00:00
wxiaoguang
20c40259f1
Fix missing memcache import (#31105)
Fix #31102
2024-05-27 13:43:32 +00:00
Rowan Bohde
c0880e7695
feat: add support for a credentials chain for minio access (#31051)
We wanted to be able to use the IAM role provided by the EC2 instance
metadata in order to access S3 via the Minio configuration. To do this,
a new credentials chain is added that will check the following locations
for credentials when an access key is not provided. In priority order,
they are:

1. MINIO_ prefixed environment variables
2. AWS_ prefixed environment variables
3. a minio credentials file
4. an aws credentials file
5. EC2 instance metadata
2024-05-27 20:56:04 +08:00
Lunny Xiao
98751108b1
Rename project board -> column to make the UI less confusing (#30170)
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.

But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
2024-05-27 08:59:54 +00:00
André Rosenhammer
14f6105ce0
Make gitea webhooks openproject compatible (#28435)
This PR adds some fields to the gitea webhook payload that
[openproject](https://www.openproject.org/) expects to exists in order
to process the webhooks.
These fields do exists in Github's webhook payload so adding them makes
Gitea's native webhook more compatible towards Github's.
2024-05-26 04:08:13 +00:00
Zettat123
7ab0988af1
Support setting the default attribute of the issue template dropdown field (#31045)
Fix #31044

According to [GitHub issue template
documentation](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema#attributes-for-dropdown),
the `default` attribute can be used to specify the preselected option
for a dropdown field.
2024-05-23 13:01:02 +00:00
wxiaoguang
fb1ad920b7
Refactor sha1 and time-limited code (#31023)
Remove "EncodeSha1", it shouldn't be used as a general purpose hasher
(just like we have removed "EncodeMD5" in #28622)

Rewrite the "time-limited code" related code and write better tests, the
old code doesn't seem quite right.
2024-05-20 15:12:50 +00:00
wxiaoguang
b6574099ed
Fix project column title overflow (#31011)
By the way:
* Re-format the "color.go" to Golang code style
* Remove unused `overflow-y: scroll;` from `.project-column` because
there is `overflow: visible`
2024-05-20 05:21:01 +00:00
wxiaoguang
339bc8bc8f
Improve reverse proxy documents and clarify the AppURL guessing behavior (#31003)
Fix #31002

1. Mention Make sure `Host` and `X-Fowarded-Proto` headers are correctly passed to Gitea
2. Clarify the basic requirements and move the "general configuration" to the top
3. Add a comment for the "container registry"
4. Use 1.21 behavior if the reverse proxy is not correctly configured

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2024-05-19 14:56:08 +00:00
wxiaoguang
821d2fc2a3
Simplify mirror repository API logic (#30963)
Fix #30921
2024-05-17 16:07:41 +00:00
dicarne
d0d6aad85f
Supports forced use of S3 virtual-hosted style (#30969)
Add a configuration item to enable S3 virtual-hosted style (V2) to solve
the problem caused by some S3 service providers not supporting path
style (V1).
2024-05-15 08:56:17 -05:00
wxiaoguang
effb405cae
Always load or generate oauth2 jwt secret (#30942)
Fix #30923
2024-05-14 14:21:38 +00:00
Zettat123
b3beaed147
Support using label names when changing issue labels (#30943)
Resolve #30917

Make the APIs for adding labels and replacing labels support both label
IDs and label names so the
[`actions/labeler`](https://github.com/actions/labeler) action can work
in Gitea.

<img width="600px"
src="https://github.com/go-gitea/gitea/assets/15528715/7835c771-f637-4c57-9ce5-e4fbf56fa0d3"
/>
2024-05-13 04:28:53 +00:00
wxiaoguang
67c1a07285
Refactor AppURL usage (#30885)
Fix #30883
Fix #29591

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2024-05-07 08:26:13 +00:00
Lunny Xiao
ebf0c96940
Move database operations of merging a pull request to post receive hook and add a transaction (#30805)
Merging PR may fail because of various problems. The pull request may
have a dirty state because there is no transaction when merging a pull
request. ref
https://github.com/go-gitea/gitea/pull/25741#issuecomment-2074126393

This PR moves all database update operations to post-receive handler for
merging a pull request and having a database transaction. That means if
database operations fail, then the git merging will fail, the git client
will get a fail result.

There are already many tests for pull request merging, so we don't need
to add a new one.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-05-07 07:36:48 +00:00
wxiaoguang
9c08637eae
Make "sync branch" also sync object format and add tests (#30878) 2024-05-06 17:02:30 +00:00
wxiaoguang
7c613f100e
Make sure git version&feature are always prepared (#30877)
Otherwise there would be more similar issues like #29287
2024-05-06 18:34:16 +02:00
wxiaoguang
c7bb3aa034
Fix markdown URL parsing for commit ID (#30812) 2024-05-04 09:48:16 +08:00
wxiaoguang
0f3e717a1a
Improve grep search (#30843)
Reduce the context line number to 1, make "git grep" search respect the
include/exclude patter, and fix #30785
2024-05-03 09:13:48 +00:00
yp05327
a50026e2f3
Fix no edit history after editing issue's title and content (#30814)
Fix #30807

reuse functions in services
2024-05-03 14:11:51 +08:00
silverwind
9235442ba5
Remove external API calls in TestPassword (#30716)
The test had a dependency on `https://api.pwnedpasswords.com` which
caused many failures on CI recently:

```
--- FAIL: TestPassword (2.37s)
    pwn_test.go:41: Get "https://api.pwnedpasswords.com/range/e6b6a": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
FAIL
coverage: 82.9% of statements
```
2024-05-02 14:43:23 +00:00
wxiaoguang
be112c1fc3
Skip gzip for some well-known compressed file types (#30796)
Co-authored-by: silverwind <me@silverwind.io>
2024-05-02 02:27:25 +00:00
wxiaoguang
ce08a9fe2f
Fix markdown rendering when mentioning users (#30795) 2024-05-02 01:00:46 +00:00
wxiaoguang
6f7cd94a02
Fix bleve fuzziness (#30799)
Fix #30797
Fix #30317
2024-05-01 15:32:52 +03:00
Chester
6709e28da7
Add API endpoints for getting action jobs status (#26673)
Sample of response, it is similar to Github actions

ref
https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#list-workflow-runs-for-a-repository

``` json
{
    "workflow_runs": [
        {
            "id": 3,
            "name": "Explore-Gitea-Actions",
            "head_branch": "main",
            "head_sha": "6d8d29a9f7a01ded8f8aeb64341cb31ee1ab5f19",
            "run_number": 3,
            "event": "push",
            "display_title": "More job",
            "status": "success",
            "workflow_id": "demo2.yaml",
            "url": "/chester/test/actions/runs/3",
            "created_at": "2023-08-22T13:41:33-04:00",
            "updated_at": "2023-08-22T13:41:37-04:00",
            "run_started_at": "2023-08-22T13:41:33-04:00"
        },
        {
            "id": 2,
            "name": "Explore-Gitea-Actions",
            "head_branch": "main",
            "head_sha": "6d8d29a9f7a01ded8f8aeb64341cb31ee1ab5f19",
            "run_number": 2,
            "event": "push",
            "display_title": "More job",
            "status": "success",
            "workflow_id": "demo.yaml",
            "url": "/chester/test/actions/runs/2",
            "created_at": "2023-08-22T13:41:30-04:00",
            "updated_at": "2023-08-22T13:41:33-04:00",
            "run_started_at": "2023-08-22T13:41:30-04:00"
        },
        {
            "id": 1,
            "name": "Explore-Gitea-Actions",
            "head_branch": "main",
            "head_sha": "e5369ab054cae79899ba36e45ee82811a6e0acd5",
            "run_number": 1,
            "event": "push",
            "display_title": "Add job",
            "status": "failure",
            "workflow_id": "demo.yaml",
            "url": "/chester/test/actions/runs/1",
            "created_at": "2023-08-22T13:15:21-04:00",
            "updated_at": "2023-08-22T13:18:10-04:00",
            "run_started_at": "2023-08-22T13:15:21-04:00"
        }
    ],
    "total_count": 3
}
```

---------

Co-authored-by: yp05327 <576951401@qq.com>
Co-authored-by: puni9869 <80308335+puni9869@users.noreply.github.com>
2024-05-01 09:40:23 +08:00
silverwind
610802df85
Fix tautological conditions (#30735)
As discovered by https://github.com/go-gitea/gitea/pull/30729.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-30 14:34:40 +02:00
wxiaoguang
61b495e5ab
Fix issue label rendering in the issue popup (#30763) 2024-04-30 02:36:32 +00:00
Chongyi Zheng
e80466f734
Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Micash
ad4e902d5a
Add support for npm bundleDependencies (#30751) 2024-04-29 16:19:06 +08:00
Chongyi Zheng
7b8e418da1
Replace deprecated math/rand functions (#30733)
Suggested by logs in #30729

- Remove `math/rand.Seed`
`rand.Seed is deprecated: As of Go 1.20 there is no reason to call Seed
with a random value.`
- Replace `math/rand.Read`
`rand.Read is deprecated: For almost all use cases, [crypto/rand.Read]
is more appropriate.`
- Replace `math/rand` with `math/rand/v2`, which is available since Go
1.22
2024-04-27 18:50:35 +02:00
wxiaoguang
d3cdef88ad
Add some tests to clarify the "must-change-password" behavior (#30693)
Follow  #30472:

When a user is created by command line `./gitea admin user create`:

Old behavior before #30472: the first user (admin or non-admin) doesn't
need to change password.

Revert to the old behavior before #30472
2024-04-27 12:23:37 +00:00
silverwind
9b2536b78f
Update misspell to 0.5.1 and add misspellings.csv (#30573)
Misspell 0.5.0 supports passing a csv file to extend the list of
misspellings, so I added some common ones from the codebase. There is at
least one typo in a API response so we need to decided whether to revert
that and then likely remove the dict entry.
2024-04-27 08:03:49 +00:00
wxiaoguang
cd70ab31cd
Fix incorrect object id hash function (#30708)
Great thanks to @oliverpool for figuring out the problem and proposing a
fix.

Regression of #28138

Incorrect hash causes the user's LFS files get all deleted when running
`doctor fix all`

(by the way, remove unused/non-standard comments)

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-26 09:49:48 +00:00
wxiaoguang
ed8c63cea3
Deduplicate lfs common code (#30704) 2024-04-26 02:53:30 +00:00
wxiaoguang
bffbbf5470
Improve oauth2 client "preferred username field" logic and the error handling (#30622)
Follow #30454
And fix #24957

When using "preferred_username", if no such field,
`extractUserNameFromOAuth2` (old `getUserName`) shouldn't return an
error. All other USERNAME options do not return such error.

And fine tune some logic and error messages, make code more stable and
more friendly to end users.
2024-04-25 11:22:32 +00:00
wxiaoguang
b79e3db264
Initial support for colorblindness-friendly themes (#30625)
Initial support for #25680

This PR only adds some simple styles from GitHub, it is big enough and
it focuses on adding the necessary framework-level supports. More styles
could be fine-tuned later.
2024-04-24 00:18:41 +08:00
Kemal Zebari
9b7af4340c
Perform Newest sort type correctly when sorting issues (#30644)
Should resolve #30642.

Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.

This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
2024-04-23 15:10:01 +08:00
wxiaoguang
e6103955cc
Fix queue test (#30646)
Fix #30643

The old test code is not stable due to the data-race described in the
TODO added at that time.

Make it stable, and remove a debug-only field from old test code.
2024-04-22 23:55:43 +00:00
silverwind
74f0c84fa4
Enable more revive linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
Lunny Xiao
e865de1e9d
Use maintained gziphandler (#30592)
Replace #27894

---------

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-04-21 00:53:45 +00:00
wxiaoguang
53cf46cae7
Fix commit file status parser (#30602)
Try to fix  #30492
2024-04-19 14:41:03 +00:00