Fix#880
Design:
1. A global setting `security.TWO_FACTOR_AUTH`.
* To support org-level config, we need to introduce a better "owner
setting" system first (in the future)
2. A user without 2FA can login and may explore, but can NOT read or
write to any repositories via API/web.
3. Keep things as simple as possible.
* This option only aggressively suggest users to enable their 2FA at the
moment, it does NOT guarantee that users must have 2FA before all other
operations, it should be good enough for real world use cases.
* Some details and tests could be improved in the future since this
change only adds a check and seems won't affect too much.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
`[repository.pull-request] DELAY_CHECK_FOR_INACTIVE_DAYS` is a new
setting to delay the mergeable check for pull requests that have been
inactive for the specified number of days.
This avoids potentially long delays for big repositories with many pull
requests. and reduces system load overall when there are many
repositories or pull requests.
When viewing the PR, checking will start immediately and the PR merge
box will automatically reload when complete. Accessing the PR through
the API will also start checking immediately.
The default value of `7` provides a balance between system load, and
keeping behavior similar to what it was before both for users and API
access. With `0` all conflict checking will be delayed, while `-1`
always checks immediately to restore the previous behavior.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Adds an API POST endpoint under `/repos/{owner}/{repo}/file-contents`
which receives a list of paths and returns a list of the contents of
these files.
This API endpoint will be helpful for applications like headless CMS
(reference: https://github.com/sveltia/sveltia-cms/issues/198) which
need to retrieve a large number of files by reducing the amount of
needed API calls.
Close#33495
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This adds a middleware for overload protection that is intended to help protect against malicious scrapers.
It does this via [`codel`](https://github.com/bohde/codel), which will perform the following:
1. Limit the number of in-flight requests to some user-defined max
2. When in-flight requests have reached their begin queuing requests.
Logged-in requests having priority above logged-out requests
3. Once a request has been queued for too long,
it has a probabilistic chance to be rejected based on how overloaded the entire system is.
When a server experiences more traffic than it can handle,
this keeps latency low for logged-in users and rejects just
enough requests from logged-out users to not overload the service.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This PR uniform all temporary directory usage so that it will be easier
to manage.
Relate to #31792
- [x] Added a new setting to allow users to configure the global
temporary directory.
- [x] Move all temporary files and directories to be placed under
os.Temp()/gitea.
- [x] `setting.Repository.Local.LocalCopyPath` now will be
`setting.TempPath/local-repo` and the customized path is removed.
```diff
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;[repository.local]
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;
-;; Path for local repository copy. Defaults to TEMP_PATH + `local-repo`, this is deprecated and cannot be changed
-;LOCAL_COPY_PATH = local-repo
```
- [x] `setting.Repository.Upload.TempPath` now will be
`settting.TempPath/uploads` and the customized path is removed.
```diff
;[repository.upload]
-;;
-;; Path for uploads. Defaults to TEMP_PATH + `uploads`
-;TEMP_PATH = uploads
```
- [x] `setting.Packages.ChunkedUploadPath` now will be
`settting.TempPath/package-upload` and the customized path is removed.
```diff
;[packages]
-;;
-;; Path for chunked uploads. Defaults it's `package-upload` under `TEMP_PATH` unless it's an absolute path.
-;CHUNKED_UPLOAD_PATH = package-upload
```
- [x] `setting.SSH.KeyTestPath` now will be
`settting.TempPath/ssh_key_test` and the customized path is removed.
```diff
[server]
-;;
-;; Directory to create temporary files in when testing public keys using ssh-keygen,
-;; default is the system temporary directory.
-;SSH_KEY_TEST_PATH =
```
TODO:
- [ ] setting.PprofDataPath haven't been changed because it may need to
be kept until somebody read it but temp path may be clean up any time.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#33966
```
;; User must sign in to view anything.
;; It could be set to "expensive" to block anonymous users accessing some pages which consume a lot of resources,
;; for example: block anonymous AI crawlers from accessing repo code pages.
;; The "expensive" mode is experimental and subject to change.
;REQUIRE_SIGNIN_VIEW = false
```
close#33086
* Add a special value for "SSH_USER" setting: `(DOER_USERNAME)`
* Improve parseRepositoryURL and add tests (now it doesn't have hard
dependency on some setting values)
Many changes are just adding "ctx" and "doer" argument to functions.
By the way, improve app.example.ini, remove all `%(key)s` syntax, it
only makes messy and no user really cares about it.
Document: https://gitea.com/gitea/docs/pulls/138
Usually enterprise/organization users would like to only allow OAuth2
login.
This PR adds a new config option to disable the password-based login
form. It is a simple and clear approach and won't block the future
login-system refactoring works.
Fix a TODO in #24821
Replace #21851Close#7633 , close#13606
Targeting issue #32271
This modification allows native Kubernetes + AWS (EKS) authentication
with the Minio client, to Amazon S3 using the IRSA role assigned to a
Service account by replacing the hard coded reference to the
`DefaultIAMRoleEndpoint` with an optional configurable endpoint.
Internally, Minio's `credentials.IAM` provider implements a discovery
flow for IAM Endpoints if it is not set.
For backwards compatibility:
- We have added a configuration mechanism for an `IamEndpoint` to retain
the unit test safety in `minio_test.go`.
- We believe existing clients will continue to function the same without
needing to provide a new config property since the internals of Minio
client also often resolve to the `http://169.254.169.254` default
endpoint that was being hard coded before
To test, we were able to build a docker image from source and, observe
it choosing the expected IAM endpoint, and see files uploaded via the
client.
Resolve#30350
The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.
Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'
This fixes#32306
---------
Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
These settings can allow users to only display the repositories explore page.
Thanks to yp05327 and wxiaoguang !
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Close#31801. Follow #31761.
Since there are so many benefits of compression and there are no reports
of related issues after weeks, it should be fine to enable compression
by default.
This will allow instance admins to view signup pattern patterns for
public instances. It is modelled after discourse, mastodon, and
MediaWiki's approaches.
Note: This has privacy implications, but as the above-stated open-source
projects take this approach, especially MediaWiki, which I have no doubt
looked into this thoroughly, it is likely okay for us, too. However, I
would be appreciative of any feedback on how this could be improved.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
close #27031
If the rpm package does not contain a matching gpg signature, the
installation will fail. See (#27031) , now auto-signing rpm uploads.
This option is turned off by default for compatibility.
We have some instances that only allow using an external authentication
source for authentication. In this case, users changing their email,
password, or linked OpenID connections will not have any effect, and
we'd like to prevent showing that to them to prevent confusion.
Included in this are several changes to support this:
* A new setting to disable user managed authentication credentials
(email, password & OpenID connections)
* A new setting to disable user managed MFA (2FA codes & WebAuthn)
* Fix an issue where some templates had separate logic for determining
if a feature was disabled since it didn't check the globally disabled
features
* Hide more user setting pages in the navbar when their settings aren't
enabled
---------
Co-authored-by: Kyle D <kdumontnu@gmail.com>
Fix#31361, and add tests
And this PR introduces an undocumented & debug-purpose-only config
option: `USE_SUB_URL_PATH`. It does nothing for end users, it only helps
the development of sub-path related problems.
And also fix#31366
Co-authored-by: @ExplodingDragon
This solution implements a new config variable MAX_ROWS, which
corresponds to the “Maximum allowed rows to render CSV files. (0 for no
limit)” and rewrites the Render function for CSV files in markup module.
Now the render function only reads the file once, having MAX_FILE_SIZE+1
as a reader limit and MAX_ROWS as a row limit. When the file is larger
than MAX_FILE_SIZE or has more rows than MAX_ROWS, it only renders until
the limit, and displays a user-friendly warning informing that the
rendered data is not complete, in the user's language.
---
Previously, when a CSV file was larger than the limit, the render
function lost its function to render the code. There were also multiple
reads to the file, in order to determine its size and render or
pre-render.
The warning: 
This PR implemented object storages(LFS/Packages/Attachments and etc.)
for Azure Blob Storage. It depends on azure official golang SDK and can
support both the azure blob storage cloud service and azurite mock
server.
Replace #25458Fix#22527
- [x] CI Tests
- [x] integration test, MSSQL integration tests will now based on
azureblob
- [x] unit test
- [x] CLI Migrate Storage
- [x] Documentation for configuration added
------
TODO (other PRs):
- [ ] Improve performance of `blob download`.
---------
Co-authored-by: yp05327 <576951401@qq.com>
We wanted to be able to use the IAM role provided by the EC2 instance
metadata in order to access S3 via the Minio configuration. To do this,
a new credentials chain is added that will check the following locations
for credentials when an access key is not provided. In priority order,
they are:
1. MINIO_ prefixed environment variables
2. AWS_ prefixed environment variables
3. a minio credentials file
4. an aws credentials file
5. EC2 instance metadata
Add a configuration item to enable S3 virtual-hosted style (V2) to solve
the problem caused by some S3 service providers not supporting path
style (V1).
Follow #30454
And fix#24957
When using "preferred_username", if no such field,
`extractUserNameFromOAuth2` (old `getUserName`) shouldn't return an
error. All other USERNAME options do not return such error.
And fine tune some logic and error messages, make code more stable and
more friendly to end users.
Initial support for #25680
This PR only adds some simple styles from GitHub, it is big enough and
it focuses on adding the necessary framework-level supports. More styles
could be fine-tuned later.
