Follow the CLI refactoring
1. Remove the "checkCommandFlags" helper
2. Unify the web startup message, make them have consistent names as `./gitea help`
3. Fine tune some other messages (see the diff)
Fix#26064
Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.
1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config
Co-authored-by: Giteabot <teabot@gitea.io>
The JSONRedirect/JSONOK/JSONError functions were put into "Base" context
incorrectly, it would cause abuse.
Actually, they are for "web context" only, so, move them to the correct
place.
And by the way, use them to simplify old code: +75 -196
Replace #21790
And close#25965 by the way (it needs a separate fix for 1.20)
Major changes:
1. Move "gitea convert" to "gitea doctor conver". The old "gitea doctor"
still works as a hidden sub-command (to avoid breaking)
2. Do not write "doctor.log" by default, it's not useful in most cases
and causes bugs like 25965
3. Improve documents
4. Fix the "help" commands. Before, the "./gitea doctor" can't show the
sub-command help correctly (regression of the last cli/v2 refactoring)
After this PR:
```
./gitea help # show all sub-commands for the app
./gitea doctor # show the sub-commands for the "doctor"
./gitea doctor help # show the sub-commands for the "doctor", as above
```
If the log file can't be opened, what should it do? panic/exit? ignore
logs? fallback to stderr?
It seems that "fallback to stderr" is slightly better than others ....
Replace #25892
Close #21942
Close #25464
Major changes:
1. Serve "robots.txt" and ".well-known/security.txt" in the "public"
custom path
* All files in "public/.well-known" can be served, just like
"public/assets"
3. Add a test for ".well-known/security.txt"
4. Simplify the "FileHandlerFunc" logic, now the paths are consistent so
the code can be simpler
5. Add CORS header for ".well-known" endpoints
6. Add logs to tell users they should move some of their legacy custom
public files
```
2023/07/19 13:00:37 cmd/web.go:178:serveInstalled() [E] Found legacy public asset "img" in CustomPath. Please move it to /work/gitea/custom/public/assets/img
2023/07/19 13:00:37 cmd/web.go:182:serveInstalled() [E] Found legacy public asset "robots.txt" in CustomPath. Please move it to /work/gitea/custom/public/robots.txt
```
This PR is not breaking.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
Replace #10912
And there are many new tests to cover the CLI behavior
There were some concerns about the "option order in hook scripts"
(https://github.com/go-gitea/gitea/pull/10912#issuecomment-1137543314),
it's not a problem now. Because the hook script uses `/gitea hook
--config=/app.ini pre-receive` format. The "config" is a global option,
it can appear anywhere.
----
## ⚠️ BREAKING ⚠️
This PR does it best to avoid breaking anything. The major changes are:
* `gitea` itself won't accept web's options: `--install-port` / `--pid`
/ `--port` / `--quiet` / `--verbose` .... They are `web` sub-command's
options.
* Use `./gitea web --pid ....` instead
* `./gitea` can still run the `web` sub-command as shorthand, with
default options
* The sub-command's options must follow the sub-command
* Before: `./gitea --sub-opt subcmd` might equal to `./gitea subcmd
--sub-opt` (well, might not ...)
* After: only `./gitea subcmd --sub-opt` could be used
* The global options like `--config` are not affected
Close#25906
Succeeded logs:
```
[I] router: completed GET /root/test/issues/posters?&q=%20&_=1689853025011 for [::1]:59271, 200 OK in 127.7ms @ repo/issue.go:3505(repo.IssuePosters)
[I] router: completed GET /root/test/pulls/posters?&q=%20&_=1689853968204 for [::1]:59269, 200 OK in 94.3ms @ repo/issue.go:3509(repo.PullPosters)
```
Before: the concept "Content string" is used everywhere. It has some
problems:
1. Sometimes it means "base64 encoded content", sometimes it means "raw
binary content"
2. It doesn't work with large files, eg: uploading a 1G LFS file would
make Gitea process OOM
This PR does the refactoring: use "ContentReader" / "ContentBase64"
instead of "Content"
This PR is not breaking because the key in API JSON is still "content":
`` ContentBase64 string `json:"content"` ``
Move `public/*` to `public/assets/*`
Some old PRs (like #15219) introduced inconsistent directory system.
For example: why the local directory "public" is accessed by
`http://site/assets`? How to serve the ".well-known" files properly in
the public directory?
For convention rules, the "public" directory is widely used for the
website's root directory. It shouldn't be an exception for Gitea.
So, this PR makes the things consistent:
* `http://site/assets/foo` means `{CustomPath}/public/assets/foo`.
* `{CustomPath}/public/.well-known` and `{CustomPath}/public/robots.txt`
can be used in the future.
This PR is also a prerequisite for a clear solution for:
* #21942
* #25892
* discourse.gitea.io: [.well-known path serving custom files behind
proxy?](https://discourse.gitea.io/t/well-known-path-serving-custom-files-behind-proxy/5445/1)
This PR is breaking for users who have custom "public" files (CSS/JS).
After getting approvals, I will update the documents.
----
## ⚠️ BREAKING ⚠️
If you have files in your "custom/public/" folder, please move them to
"custom/public/assets/".
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Giteabot <teabot@gitea.io>
The "creating dir/file during load config" is a longstanding and complex
problem.
This PR only does a quick patch, it still needs more refactorings in the
future.
Fix#25938
Before, in dev mode, there might be some error logs like:
```
2023/07/17 13:54:51 ...s/assetfs/layered.go:221:WatchLocalChanges() [E] Unable to watch directory .: lstat /data/work/gitea/custom/templates: no such file or directory
```
Because there is no "custom/templates" directory.
After: ignore such error, no such error message anymore.
Related issue: #18368
It doesn't seem right to "guess" the file encoding/BOM when using API to
upload files.
The API should save the uploaded content as-is.
Fix#25726#17846 chose an incorrect WORK_DIR path for docker root image.
Gitea's work-path was already used as the base path for various paths
(like AppDataPath), so, the work-path should be mounted to a volume in a
docker image.
Now, for docker root image, it's unavoidable to mix the
WorkPath/CustomPath/AppDataPath in the same directory ("/data/gitea"),
because some of them have already been mixed.
Some directories in the screenshot are for "CustomPath" , while others
are for "AppDataPath", due to the technical debts in old code:
```
CUSTOM_PATH="/data/gitea"
APP_DATA_PATH = /data/gitea
```
<details>
</details>
This PR is breaking but this is the only way at the moment to avoid
users losing their data accidently
Co-authored-by: Giteabot <teabot@gitea.io>
Replace #25580Fix#19453
The problem was: when users set "GITEA__XXX__YYY" , the "install page"
doesn't respect it.
So, to make the result consistent and avoid surprising end users, now
the "install page" also writes the environment variables to the config
file.
And, to make things clear, there are enough messages on the UI to tell
users what will happen.
There are some necessary/related changes to `environment-to-ini.go`:
* The "--clear" flag is removed and it was incorrectly written there.
The "clear" operation should be done if INSTALL_LOCK=true
* The "--prefix" flag is removed because it's never used, never
documented and it only causes inconsistent behavior.
Fix#25627
1. `ctx.Data["Link"]` should use relative URL but not AppURL
2. The `data-params` is incorrect because it doesn't contain "page". JS
can simply use "window.location.search" to construct the AJAX URL
3. The `data-xxx` and `id` in notification_subscriptions.tmpl were
copied&pasted, they don't have affect.
To mock a handler:
```go
web.RouteMock(web.MockAfterMiddlewares, func(ctx *context.Context) {
// ...
})
defer web.RouteMockReset()
```
It helps:
* Test the middleware's behavior (assert the ctx.Data, etc)
* Mock the middleware's behavior (prepare some context data for handler)
* Mock the handler's response for some test cases, especially for some
integration tests and e2e tests.
The code was just copied&pasted, it causes problems now.
There are a lot (for every package) broken translations. eg:
```
# en-US
conda.documentation = For more information on the Conda registry, see
<a target="_blank" rel="noopener noreferrer" href="%s">the documentation</a>.
# fr-FR (and many languages)
conda.documentation=Pour plus d'informations sur le registre Conda, voir
<a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/fr-fr/packages/conda/">la documentation</a>.
```
To resolve the problem fundamentally, use a general string, and trigger
the re-translating on Crowdin side.
And, it should really really really avoid introducing too much
copied&pasted code .......
Remove unnecessary `if opts.Logger != nil` checks.
* For "CLI doctor" mode, output to the console's "logger.Info".
* For "Web Task" mode, output to the default "logger.Debug", to avoid
flooding the server's log in a busy production instance.
Co-authored-by: Giteabot <teabot@gitea.io>
Replace #25446, fix#25438
All "cancel" buttons which do not have "type" should not submit the
form, should not be triggered by "Enter".
This is a complete fix for all modal dialogs.
The major change is "modules/aria/modal.js", "devtest" related code is
for demo/test purpose.
More fix for #24981
* #24981Close#22361
* #22361
There were many patches for Gitea's sub-commands to satisfy the facts:
* Some sub-commands shouldn't output any log, otherwise the git protocol
would be broken
* Sometimes the users want to see "verbose" or "quiet" outputs
That's a longstanding problem, and very fragile. This PR is only a quick
patch for the problem.
In the future, the sub-command system should be refactored to a clear
solution.
----
Other changes:
* Use `ReplaceAllWriters` to replace
`RemoveAllWriters().AddWriters(writer)`, then it's an atomic operation.
* Remove unnecessary `syncLevelInternal` calls, because
`AddWriters/addWritersInternal` already calls it.
Co-authored-by: Giteabot <teabot@gitea.io>
A regression of #25330 : The nil "Action" should be treated as "help"
In old releases: `./gitea admin` show helps
After #25330: `./gitea admin` panics (although the code returned `nil`
if action is nil, but Golang's quirk is: nil in interface is not nil)
With this PR: `./gitea admin` shows helps as the old releases.
Fix#25481
The `InitWorkPathAndCommonConfig` calls `LoadCommonSettings` which does
many checks like "current user is root or not".
Some commands like "environment-to-ini" shouldn't do such check, because
it might be run with "root" user at the moment (eg: the docker's setup
script)
ps: in the future, the docker's setup script should be improved to avoid
Gitea's command running with "root"
Close#20976Close#20975
1. Fix the bug: the TOC in footer was incorrectly rendered as main
content's TOC
2. Fix the layout: on mobile, the TOC is put above the main content,
while the sidebar is put below the main content
3. Auto collapse the TOC on mobile
ps: many styles of "wiki.css" are moved from old css files, so leave
nits to following PRs.
Fix #25438
All non-"ok" buttons which do not have "type" should not submit the
form, should not be triggered by "Enter".
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
In modern days, there is no reason to make users set "charset" anymore.
Close#25378
## ⚠️ BREAKING
The key `[database].CHARSET` was removed completely as every newer
(>10years) MySQL database supports `utf8mb4` already.
There is a (deliberately) undocumented new fallback option if anyone
still needs to use it, but we don't recommend using it as it simply
causes problems.
# The problem
There were many "path tricks":
* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
* The "serv" process started by OpenSSH server
* The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling
# The solution
* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.
The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default
The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default
## ⚠️ BREAKING
If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.
----
Close#24818Close#24222Close#21606Close#21498Close#25107Close#24981
Maybe close#24503
Replace #23301
Replace #22754
And maybe more
That's a longstanding INI package problem: the "MustXxx" calls change
the option values, and the following "Save" will save a lot of garbage
options into the user's config file.
Ideally we should refactor the INI package to a clear solution, but it's
a huge work.
A clear workaround is what this PR does: when "Save", load a clear INI
instance and save it.
Partially fix#25377, the "install" page needs more fine tunes.
A regression of #25210
The `e.target` is not "this", eg: `<button link-action><svg></button>`,
then `this` should be `button` but `e.target` is `svg`.
I will propose a clearer and complete solution for these "link-action"
"show-modal" elements after #24724
Co-authored-by: Giteabot <teabot@gitea.io>
If the APP_DATA_PATH isn't written into the config when installing, then
its value is uncertain because some Gitea command doesn't run with
correct WorkPath.
This is a quick fix for #25330 and can be backported.
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
package.
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Fixes: https://github.com/go-gitea/gitea/issues/25282
Fix the problems:
1. The `repo-button-row` had various patches before, this PR makes it
consistent
2. The "Add File" has wrong CSS class "icon", remove it
3. The "Add File" padding was overridden by "!important", fix it by
`.repo-button-row .button.dropdown` with comment
4. The selector `.ui.segments ~ .ui.top.attached.header` is incorrect,
it should use `+`
It causes not only one issue like #25221 (the footer width was also
affected by that change and was fixed some time ago)
The problem of "overflow: overlay" (#21850) is:
* It's not widely supported and is non-standard
https://caniuse.com/css-overflow-overlay
* It's not widely tested in Gitea (some standard layout like `ui
container + ui grid` may break it).
* The benefit seems smaller than the problems it brings.
So, I think it is good to revert it.
----
Let's leave enough time for testing and reviewing.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
Clarify the "link-action" behavior:
> // A "link-action" can post AJAX request to its "data-url"
> // Then the browser is redirect to: the "redirect" in response, or
"data-redirect" attribute, or current URL by reloading.
And enhance the "link-action" to support showing a modal dialog for
confirm. A similar general approach could also help PRs like
https://github.com/go-gitea/gitea/pull/22344#discussion_r1062883436
> // If the "link-action" has "data-modal-confirm(-html)" attribute, a
confirm modal dialog will be shown before taking action.
And a lot of duplicate code can be removed now. A good framework design
can help to avoid code copying&pasting.
---------
Co-authored-by: silverwind <me@silverwind.io>
According to my test, the UI (emoji) is fine in Safari
And actually the code is just dead code, because the "resize" event is
never fired on page loading. So for most cases users just view the pages
without this hacky patch, nobody ever complains.
That ID is a "copy&paste" error, it conflicts with the
`initRepoMigrationStatusChecker` logic, which is the right function for
a real `#repo_migrating` element. That wrong ID causes incorrect page
navigation after installation.
Follow:
* #22697
There are some bugs in #22697:
* https://github.com/go-gitea/gitea/pull/22697#issuecomment-1577957966
* the webauthn failure message is never shown and causes console error
* The `document.getElementById('register-button')` and
`document.getElementById('login-button')` is wrong
* there is no such element in code
* it causes JS error when a browser doesn't provide webauthn
* the end user can't see the real error message
These bugs are fixed in this PR.
Other changes:
* Use simple HTML/CSS layouts, no need to use too many `gt-` patches
* Make the webauthn page have correct "page-content" layout
* The "data-webauthn-error-msg" elements are only used to provide locale
texts, so move them into a single "gt-hidden", then no need to repeat a
lot of "gt-hidden" in code
* The `{{.CsrfTokenHtml}}` is a no-op because there is no form
* Many `hideElem('#webauthn-error')` in code is no-op because the
`webauthn-error` already has "gt-hidden" by default
* Make the tests for "URLEncodedBase64" really test with concrete cases.
Screenshots:
* Error message when webauthn fails (before, there is no error message):
<details>
</details>
* Error message when webauthn is unavailable
<details>
</details>
Before, Gitea shows the database table stats on the `admin dashboard`
page.
It has some problems:
* `count(*)` is quite heavy. If tables have many records, this blocks
loading the admin page blocks for a long time
* Some users had even reported issues that they can't visit their admin
page because this page causes blocking or `50x error (reverse proxy
timeout)`
* The `actions` stat is not useful. The table is simply too large. Does
it really matter if it contains 1,000,000 rows or 9,999,999 rows?
* The translation `admin.dashboard.statistic_info` is difficult to
maintain.
So, this PR uses a separate page to show the stats and removes the
`actions` stat.
## ⚠️ BREAKING
The `actions` Prometheus metrics collector has been removed for the
reasons mentioned beforehand.
Please do not rely on its output anymore.
The INI package has many bugs and quirks, and in fact it is
unmaintained.
This PR is the first step for the INI package refactoring:
* Use Gitea's "config_provider" to provide INI access
* Deprecate the INI package by golangci.yml rule
The admin config page has been broken for many many times, a little
refactoring would make this page panic.
So, add a test for it, and add another test to cover the 500 error page.
Co-authored-by: Giteabot <teabot@gitea.io>
gitea.com experienced the corrupted LevelQueue bug again.
I think the problem is clear now: if the keys in LevelDB went
out-of-sync, the LevelQueue itself doesn't have the ability to recover,
eg:
* LevelQueue.Len() reports 100
* LevelQueue.LPop() reports ErrNotFound = errors.New("no key found")
So it needs to dive into the LevelDB to remove all keys to recover the
corrupted LevelQueue.
More comments are in TestCorruptedLevelQueue.
The old EventWriter's Run does:
```go
for {
handlePause()
select {
case event <- Queue:
write the log event ...
}
}
```
So, if an event writer is started before the logger is paused, there is
a chance that the logger isn't paused for the first message.
The new logic is:
```go
for {
select {
case event <- Queue:
handlePause()
write the log event ...
}
}
```
Then the event writer can be correctly paused
The old process manager's `Trace` function by default calls `log.Printf`
to output "trace" level logs. That's not ideal because by default the
trace level logs should not be outputted. In history it didn't cause
problems because there was no other call to the process manager before
the logger system's initialization.
But if there is any package using the process manager before the "Trace"
function gets assigned to the logger system's trace function, the
process manager will outputs unexpected verbose messages, this behavior
is not expected in most cases.
Now, the logger system also uses process manager to manage its goroutine
contexts, so it's the time to fix the old "trace" behavior: by default,
do not output the trace level messages. Fix#24951
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.
The old queue system doesn't work well with context, so the old queues
need it.
After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.
This PR also fixes some legacy queue-init problems, eg:
* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic
After this PR, many goroutines could have better display name:
1. non-dev mode is treated as prod mode, to protect users from
accidentally running in dev mode if there is a typo in this value.
2. in dev mode, do not need to really exit if there are template errors,
because the template errors could be fixed by developer soon and the
templates get reloaded, help:
* https://github.com/go-gitea/gitea/issues/24845#issuecomment-1557615382
3. Fine tune the mail template loading message.
Fix#24896
If users set different languages by `linguist-language`, the `stats` map
could be: `java: 100, Java: 200`.
Language stats are stored as case-insensitive in database and there is a
unique key.
So, the different language names should be merged to one unique name:
`Java: 300`
Replace #19857Close#19856Close#10311Close#10123
Major changes:
1. Move a lot of code from `environment-to-ini.go` to `config_env.go` to
make them testable.
2. Add `__FILE` support
3. Update documents
4. Add tests
wxiaoguangandGitHub
