blob: bedc4fb834187e5348130768949de3a62a32a4ce (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
|
# Mirror
Mirror free and open-source projects you like with minimal effort.
Use this project as a service to keep a local or hosted copy of the projects
source code or their assets.
## Status
This project is in pre-alpha and under active development. Expect rapid and
breaking changes.
Planned improvements and known bugs are listed on the project's
[todo](https://git.server.ky/slackcoder/mirror/tree?h=todo) branch.
## Requirements
The following applications are required on your system.
- Golang >= 1.22.4
- git
- rsync
## Verification
Verify the project and its assets using the GPG Key included in the project.
```
gpg --import GPG-KEY
git commit verify-commit HEAD
```
To verify the projects vendored dependencies:
```
go mod verify
```
## Building and Installation
### Slackware
You can create a package using the Slackbuild script provided with the project.
```
cd contrib/slackbuild
sudo sh *.SlackBuild
```
### Other Unix-like systems
Standard practice is to install under /usr/local/sbin
```
go build ./cmd/...
sudo mv mirror /usr/local/sbin
sudo chmod +x /usr/local/sbin/mirror
```
You will then need to integrate it into your system's service management to
your liking.
## Configuration
Configuration is provided via [TOML](https://toml.io) file format.
Refer to the following example. Mirror will mirror each project at a random
interval between 1 and 24 hours.
```
[global]
# The minimum time to wait before mirroring.
min_interval = "1h"
# The maximum time to wait before mirroring.
max_interval = "24h"
[[mirrors]]
method = "rsync"
from = "rsync://mirrors.kernel.org/slackware/slackware64-current"
to = "/mirror/slackware/slackware64-current"
[[mirrors]]
method = "git"
from = "https://github.com/ytdl-org/youtube-dl"
to = "/srv/git/slackcoder/youtube-dl"
description = "Command-line program to download videos from YouTube.com and other video sites"
[[mirrors]]
method = "github-assets"
from = "https://github.com/ytdl-org/youtube-dl"
to = "/mirror/youtube-dl"
```
Configuration may also be split across files in a directory. By default
loads configuration from /etc/mirror/mirror.toml and the /etc/mirror/conf.d
directory.
## Staging and Verification (Experimental)
You can ensure mirror integrity by verifying the project in a staging directory
before saving it.
Use the 'verify' parameter to define Bash executed shell commands. A non-zero
exit value is considered a verification failure. The standard error output is
then written to the log.
Use the 'staging-method' and 'staging-path' parameters to customize how the
incoming mirror version is stored. Both these values can be defined globally
or specific to a mirror.
The possible 'staging-method' values are:
- none: Save the data directly to the mirror destination.
- temporary: Save the data in a directory to be deleted after verification.
- persistent: Use a second location to store incoming data without clearing it after.
The 'staging-path' defines the parent directory for where staging occurs, by default it is '/tmp/mirror'.
```
[[mirrors]]
method = "rsync"
from = "rsync://mirrors.kernel.org/slackware/slackware64-15.0"
to = "/mirror/slackware/slackware64-15.0"
staging-path = "/tmp/mirror"
staging = "persistent"
verify = """
(gpg --verify-files *.asc && tail +13 CHECKSUMS.md5 | md5sum -c --quiet -) && \
(find slackware64 -print0 -name '*.asc' | xargs -0L1 gpg2 --verify-files)
"""
```
|
