1
1
mirror of https://github.com/go-gitea/gitea synced 2024-12-23 17:14:27 +00:00
Commit Graph

46 Commits

Author SHA1 Message Date
KN4CK3R
440039c0cc
Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
KN4CK3R
c03e488e14
Add LFS Migration and Mirror (#14726)
* Implemented LFS client.

* Implemented scanning for pointer files.

* Implemented downloading of lfs files.

* Moved model-dependent code into services.

* Removed models dependency. Added TryReadPointerFromBuffer.

* Migrated code from service to module.

* Centralised storage creation.

* Removed dependency from models.

* Moved ContentStore into modules.

* Share structs between server and client.

* Moved method to services.

* Implemented lfs download on clone.

* Implemented LFS sync on clone and mirror update.

* Added form fields.

* Updated templates.

* Fixed condition.

* Use alternate endpoint.

* Added missing methods.

* Fixed typo and make linter happy.

* Detached pointer parser from gogit dependency.

* Fixed TestGetLFSRange test.

* Added context to support cancellation.

* Use ReadFull to probably read more data.

* Removed duplicated code from models.

* Moved scan implementation into pointer_scanner_nogogit.

* Changed method name.

* Added comments.

* Added more/specific log/error messages.

* Embedded lfs.Pointer into models.LFSMetaObject.

* Moved code from models to module.

* Moved code from models to module.

* Moved code from models to module.

* Reduced pointer usage.

* Embedded type.

* Use promoted fields.

* Fixed unexpected eof.

* Added unit tests.

* Implemented migration of local file paths.

* Show an error on invalid LFS endpoints.

* Hide settings if not used.

* Added LFS info to mirror struct.

* Fixed comment.

* Check LFS endpoint.

* Manage LFS settings from mirror page.

* Fixed selector.

* Adjusted selector.

* Added more tests.

* Added local filesystem migration test.

* Fixed typo.

* Reset settings.

* Added special windows path handling.

* Added unit test for HTTPClient.

* Added unit test for BasicTransferAdapter.

* Moved into util package.

* Test if LFS endpoint is allowed.

* Added support for git://

* Just use a static placeholder as the displayed url may be invalid.

* Reverted to original code.

* Added "Advanced Settings".

* Updated wording.

* Added discovery info link.

* Implemented suggestion.

* Fixed missing format parameter.

* Added Pointer.IsValid().

* Always remove model on error.

* Added suggestions.

* Use channel instead of array.

* Update routers/repo/migrate.go

* fmt

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-08 18:25:57 -04:00
Lunny Xiao
7b2ff82d20 Move some repository methods from models to modules/repository (#9353)
* Move some repository methods from models to modules/repository

* fix test
2019-12-14 18:30:01 +01:00
Lunny Xiao
d151503d34 Upgrade xorm to v0.8.0 (#8536) 2019-10-17 11:26:49 +02:00
Lunny Xiao
7ff783b732
Move mirror to a standalone package from models (#7486)
* move mirror to a standalone package

* fix mirror address in template

* fix tests

* fix lint

* fix comment

* fix tests

* fix tests

* fix vendor

* fix fmt

* fix lint

* remove wrong file submitted

* fix conflict

* remove unrelated changes

* fix go mod

* fix tests

* clean go mod

* make vendor work

* make vendor work

* fix tests

* remove duplicated test
2019-10-01 21:40:17 +08:00
Lunny Xiao
4959bf1c09
Move create release from models to a standalone package (#7539)
* move create release from models to a standalone package

* fix lint

* fix comment year

* fix lint

* fix lint

* fix package import name

* fix vendor

* fix go mod

* some refactors

* fix vendor

* use go1.12 make vendor

* fix vendor
2019-09-15 23:28:25 +08:00
Tamal Saha
171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
zeripath
bee1227b2f
Extract the username and password from the mirror url (#7651)
* Explode out mirror username and password

* Update models/repo_mirror.go

* Just roundtrip the password

* remove unused declaration

* Update templates/repo/settings/options.tmpl
2019-08-16 22:56:57 +01:00
Lunny Xiao
85202d4784
Display ui time with customize time location (#7792)
* display ui time with customize time location

* fix lint

* rename UILocation to DefaultUILocation

* move time related functions to modules/timeutil

* fix tests

* fix tests

* fix build

* fix swagger
2019-08-15 22:46:21 +08:00
Lunny Xiao
d2958d9f46 Move status table to cron package (#7370) 2019-07-15 20:13:03 -04:00
Lunny Xiao
edc94c7041 Monitor all git commands; move blame to git package and replace git as a variable (#6864)
* monitor all git commands; move blame to git package and replace git as a variable

* use git command but not other commands

* fix build

* move exec.Command to git.NewCommand

* fix fmt

* remove unrelated changes

* remove unrelated changes

* refactor IsEmpty and add tests

* fix tests

* fix tests

* fix tests

* fix tests

* remove gitLogger

* fix fmt

* fix isEmpty

* fix lint

* fix tests
2019-06-26 14:15:26 -04:00
zeripath
57b2ce03d5
Handle early git version's lack of get-url (#7065) 2019-05-29 06:49:08 +01:00
zeripath
159294f799 When mirroring we should set the remote to mirror (#6824) 2019-05-02 09:43:20 +03:00
zeripath
b3e757a06c
Correctly adjust mirror url (#6593) 2019-04-12 21:52:57 +01:00
zeripath
704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
Lunny Xiao
d578b71d61
move code.gitea.io/git to code.gitea.io/gitea/modules/git (#6364)
* move code.gitea.io/git to code.gitea.io/gitea/modules/git

* fix imports

* fix fmt

* fix misspell

* remove wrong tests data

* fix unit tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* enable Debug to trace the failure tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* comment commit count tests since git clone depth is 50

* fix tests

* update from code.gitea.io/git

* revert change to makefile
2019-03-27 17:33:00 +08:00
Jonas Bröms
599adde1bc Add option to disable automatic mirror syncing. (#5242)
Setting the interval to 0 will disable to automatic syncing.
2018-11-08 18:58:02 -05:00
Lauris BH
fa4663e61e Add push webhook support for mirrored repositories (#4127) 2018-09-06 22:06:09 -04:00
Lanre Adelowo
33bc2ebdfa Make sure to reset commit count in the cache on mirror syncing (#4720)
* Make sure to reset commit count in the cache on mirror syncing

* reset count of commits in all branches
2018-08-23 14:15:07 +08:00
Bo-Yi Wu
845ba9d153 fix: if Mirrors repo no content is fetched, updated time should not b… (#3551)
* fix: if Mirrors repo no content is fetched, updated time should not be changed

* fix: sync update time from mirror repo.

* fix: one single session.

* update comment.

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
2018-02-22 18:27:09 -06:00
Lunny Xiao
f2e20c81b6 Refactor struct's time to remove unnecessary memory usage (#3142)
* refactor struct's time to remove unnecessary memory usage

* use AsTimePtr simple code

* fix tests

* fix time compare

* fix template on gpg

* use AddDuration instead of Add
2017-12-11 06:37:04 +02:00
Ethan Koenig
3c1b1ca78e Fix error message sanitiziation (#3082) 2017-12-04 03:48:03 +02:00
Ethan Koenig
a0964775ab Sanitize logs for mirror sync (#3057) 2017-12-03 13:29:41 +08:00
Jonas Bröms
847272a70d Only update mirror last update after successful sync (#2730) 2017-10-18 21:50:41 +03:00
Ethan Koenig
aa962deec0 Replace deprecated Id method with ID (#2655) 2017-10-05 07:43:04 +03:00
Lunny Xiao
a8717e5e3a Use AfterLoad instead of AfterSet on Structs (#2628)
* use AfterLoad instead of AfterSet on Structs

* fix the comments on AfterLoad

* fix the comments on action AfterLoad
2017-10-01 19:52:35 +03:00
Lauris BH
7a0297819d Sync releases table with tags on push and for mirrors (#2459)
* Sync releases table with tags on push and for mirrors

* Code style fixes

* Fix api to return only releases

* Optimize release creation and update
Minimize posibility of race conditions

* Fix release lower tag name updating

* handle tag reference update by addionally comparing commit id
2017-09-20 13:26:49 +08:00
Lunny Xiao
005900baea Use created & updated instead BeforeInsert & BeforeUpdate (#2482)
* use created & updated instead BeforeInsert & BeforeUpdate

* fix vendor checksum

* only show generated SQL when development mode

* remove extra update column updated_unix

* remove trace config
2017-09-13 08:18:22 +03:00
Ethan Koenig
bfb44f8854 Fix status table race condition (#1835) 2017-05-31 16:57:17 +08:00
Jonas
be6edaddcb Repo size in admin panel (#1482)
* Implementation of the feature to view repository size in admin panel
 * Move GetRepoSize to git module
 * Repository.RepoSize -> Repository.Size
 * RepoSize -> Size in template
 * Redo a few bits and pieces
 * Update size when syncing mirror or forking
 * Remove GetRepoSize
 * Changed fatal errors to error message

* Copy migration code from Gogs

* make fmt
2017-04-11 21:30:15 +08:00
Jonas
54f0293f0a Mirror sync interval specified as duration string (#1407)
* Sync interval specifed as duration string

* Changed mirror interval text

* make fmt

* Add MinInterval for mirror sync

* Use duration internally

* Changed min default to 10m

* make fmt

* Incorrect default

* Removed defaults in MustDuration()

* Add Mirror interval migration

* Default values corrected

* Use transaction during migration

* Change http 500 to page with error message

* Cleanup session.commit()
2017-04-08 23:27:26 +08:00
Jonas
e7493e953f Corrected Mirror.NextUpdate not set (#1388) 2017-04-05 17:56:45 -07:00
Gabriel Jackson
bf6f61cc69 Cleanup log messaging
This change corrects a few logging issues:

 * Standardized formatting errors with '%v'.
 * Standardized failure warning word usage.
 * Corrected an instance of using the standard log library when
   the gitea log library should be used instead.
2017-02-02 15:24:18 +01:00
Matthias Loibl
d1006150fb Refactor process package and introduce ProcessManager{} with tests (#75)
* Add a process.Manager singleton with process.GetManager()

* Use process.GetManager everywhere

* Fix godoc comments for process module

* Increment process counter id after locking the mutex
2017-01-17 13:58:58 +08:00
Andrey Nering
84b7d29d34 Create missing database indexes (#596) 2017-01-06 23:14:33 +08:00
Ethan Koenig
1207bda94b Fix typos in models/ (#576) 2017-01-05 08:50:34 +08:00
Alexander Lunegov
d5d21b67d2 Fix string format verbs (#3637) 2016-12-22 08:18:41 +01:00
Bwko
7bf7042013 Lint models/repo_mirror.go 2016-11-26 11:23:55 +01:00
Thibault Meyer
b0ddced2b5
Merge remote-tracking branch 'upstream/master' into feature/rewrite-xorm-queries
# Conflicts:
#	models/git_diff.go
#	models/issue.go
#	models/org.go
#	models/pull.go
#	models/repo.go
2016-11-10 21:05:52 +01:00
Thibault Meyer
a4454f5d0f
Rewrite XORM queries 2016-11-10 20:59:51 +01:00
Sandro Santilli
4247304f5a Update import paths from github.com/go-gitea to code.gitea.io (#135)
- Update import paths from github.com/go-gitea to code.gitea.io
- Fix import path for travis

See https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-11-10 17:24:48 +01:00
Sandro Santilli
b3828e38a5 more 2016-11-07 17:41:28 +01:00
Rémy Boulanouar
2d68bd1ef9 Change import reference to match gitea instead of gogs (#37) 2016-11-03 10:29:56 -02:00
Unknwon
6da55159a2
#3589 LoadRepoConfig after ORM is initialized 2016-09-02 23:18:37 -04:00
Unknwon
c2afdf2192 Minor code fix [CI SKIP] 2016-08-31 04:31:53 -07:00
Unknwon
8516dfcb6c #2018 able to sync now for mirrors
- Refactor code to use sync.UniqueQueue
- Closes #3509
2016-08-30 16:18:33 -07:00