README.md 21.4 KB
Newer Older
Aral Balkan's avatar
Aral Balkan committed
1
# Indie Web Server
Aral Balkan's avatar
Aral Balkan committed
2

Aral Balkan's avatar
Aral Balkan committed
3
![Screenshot of Indie Web Server in use](images/indie-web-server-8.0.0.jpeg)
4

5
__Indie Web Server is a secure and seamless [Small Tech](https://ar.al/2019/03/04/small-technology/) personal web server for Linux and Linux-like* operating systems.__
Aral Balkan's avatar
Aral Balkan committed
6

Aral Balkan's avatar
Aral Balkan committed
7 8
  - Zero-configuration – It Just Works 🤞™.

Aral Balkan's avatar
Aral Balkan committed
9
  - Develop with automatically-provisioned locally-trusted TLS courtesy of [mkcert](https://github.com/FiloSottile/mkcert) seamlessly integrated via [Nodecert](https://source.ind.ie/hypha/tools/nodecert).
Aral Balkan's avatar
Aral Balkan committed
10

Aral Balkan's avatar
Aral Balkan committed
11 12
  - Test and deploy with automatically-provisioned globally-trusted TLS courtesy of [Let’s Encrypt](https://letsencrypt.org/) seamlessly integrated via [ACME TLS](https://source.ind.ie/hypha/tools/acme-tls) and [systemd](https://freedesktop.org/wiki/Software/systemd/). Your server will score an A on the [SSL Labs SSL Server Test](https://www.ssllabs.com/ssltest).

Aral Balkan's avatar
Aral Balkan committed
13
  <ins>Note:</ins> Live deployments via startup daemons are only supported on Linux distributions with systemd.
Aral Balkan's avatar
Aral Balkan committed
14

15 16
  \* Works with Linux, macOS, and Windows Subsystem for Linux.

17
## Install
Aral Balkan's avatar
Aral Balkan committed
18

Aral Balkan's avatar
Aral Balkan committed
19 20
Copy and paste the following commands into your terminal:

21
### Native binaries
Aral Balkan's avatar
Aral Balkan committed
22

23
__Before you pipe any script into your computer, always [view the source code](https://ind.ie/web-server/install.sh) and make sure you understand what it does.__
Aral Balkan's avatar
Aral Balkan committed
24 25

```
26
wget -qO- https://ind.ie/web-server/install.sh | bash
Aral Balkan's avatar
Aral Balkan committed
27 28 29 30
```

### Node.js

Aral Balkan's avatar
Aral Balkan committed
31
```sh
Aral Balkan's avatar
Aral Balkan committed
32
npm i -g @ind.ie/web-server
Aral Balkan's avatar
Aral Balkan committed
33 34
```

35
## Use
36

37
### Local
38

39
Start serving the current directory at https://localhost as a regular process using locally-trusted certificates:
40 41

```shell
Aral Balkan's avatar
Aral Balkan committed
42
$ web-server
43 44
```

45 46
### Proxy server (local)

Aral Balkan's avatar
Aral Balkan committed
47
You can use Indie Web Server as a development-time reverse proxy for HTTP and WebSocket connections. For example, if you use [Hugo](https://gohugo.io/) and you’re running `hugo server` on the default HTTP port 1313. You can run a HTTPS reverse proxy at https://localhost [with LiveReload support](https://source.ind.ie/hypha/tools/web-server/blob/master/bin/web-server.js#L237) using:
48 49

```shell
50
$ web-server proxy localhost:1313
51 52
```

53 54 55 56
This will create and serve the following proxies:

  * http://localhost:1313 → https://localhost
  * ws://localhost:1313 → wss://localhost
57

Aral Balkan's avatar
Aral Balkan committed
58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83
### Live sync to remote server

Part of local development involves deploying your changes to a live server at some point. You can use Indie Web Server to handle this for you in real-time:

```shell
$ web-server sync my-demo/ my-demo.site
```

The above command will start a local development server at _https://localhost_. Additionally, it will watch the folder _my-demo_ for changes and sync any changes via rsync over ssh to _my-demo.site_. Without any customisations, the sync command assumes that your account on your remote server has the same name as your account on your local machine and that the folder you are watching (_my-demo_, in the example above) is located at _/home/your-account/my-demo_. You can change these defaults using optional arguments.

```shell
$ web-server sync my-folder/ --host=my-demo.site --account=a-different-account --folder=not-my-folder
```

e.g., The above command will watch the the contents of the _my-folder_ directory and sync it to _a-different-account@my-demo.site:/home/a-different-account/not-my-folder_.

You can also customise the destination folder completely but supplying a custom remote connection string using the `--to` option:

```shell
$ web-server sync my-folder/ --to=some-account@my-demo.site:/var/www
```

Like the other commands, if you do not specify a folder, the current folder will be used by default.

__Important:__ The trailing slash is important. It means “sync the contents of this folder but not the folder itself”. If you leave out the trailing slash, it means “sync the contents of this folder and the folder itself”. The latter will result in a folder of the same name as your local folder being created in your destination folder on the remote server. Indie Web Server inherits this functionality from the rsync command itself and keeps it for consistency.

84 85 86 87 88 89
__Note:__ If you want to carry out a one-time sync and not run the server afterwards, use the `--exit-on-sync` flag. e.g.,

```shell
$ web-server sync my-folder my-demo.site --exit-on-sync
```

90 91
### Global (ephemeral)

Aral Balkan's avatar
Aral Balkan committed
92
Start serving the _site_ directory at your _hostname_ as a regular process using globally-trusted Let’s Encrypt certificates:
93 94

```shell
95
$ web-server global site
96 97
```

98
Then use, for example, [ngrok](https://ngrok.com/) (Pro+) to point a custom domain name to your temporary staging server. Make sure you set your `hostname` file (e.g., in `/etc/hostname` or via `hostnamectl set-hostname <hostname>` or the equivalent for your platform) to match your domain name. The first time you hit your server via your hostname it will take a little longer to load as your Let’s Encrypt certificates are being automatically provisioned by ACME TLS.
99

Aral Balkan's avatar
Aral Balkan committed
100
When you start your server using the `global` command, it will run as a regular process. It will not be restarted if it crashes or if you exit the foreground process or restart the computer.
101 102 103

### Global (persistent)

104
__Available on Linux distributions with systemd (most Linux distributions, but [not these ones](https://sysdfree.wordpress.com/2019/03/09/135/) or on macOS).__
105

Aral Balkan's avatar
Aral Balkan committed
106
Start serving the _site_ directory at your _hostname_ as a daemon that is automatically run at system startup and restarted if it crashes:
107 108

```shell
109
$ web-server enable site
110 111
```

112
The `enable` command sets up your server to start automatically when your server starts and restart automatically if it crashes. Requires superuser privileges on first run to set up the launch item.
113

114
For example, if you run the command on a connected server that has the ar.al domain pointing to it and `ar.al` set in _/etc/hostname_, you will be able to access the site at https://ar.al. The first time you hit it, it will take a little longer to load as your Let’s Encrypt certificates are being automatically provisioned by ACME TLS.
115

116
When the server is enabled, you can also use:
Aral Balkan's avatar
Aral Balkan committed
117

118
  - `disable`: Stop server and remove from startup.
119
  - `logs`: Display and tail server logs.
120
  - `status`: Display detailed server information (press ‘q’ to exit).
Aral Balkan's avatar
Aral Balkan committed
121

122
Indie Web Server uses the [systemd](https://freedesktop.org/wiki/Software/systemd/) to start and manage the daemon. Beyond the commands listed above that Indie Web Server supports natively (and proxies to systemd), you can make use of all systemd functionality via the `systemctl` and `journalctl` commands.
Aral Balkan's avatar
Aral Balkan committed
123

124 125
## Build and test from source

Aral Balkan's avatar
Aral Balkan committed
126 127
### Global Node.js module

128 129 130 131
```shell
# Clone and install.
git clone https://source.ind.ie/hypha/tools/web-server.git
cd web-server
132 133
npm i         # Install modules and development dependencies.
npm i -g .    # Install globally for access to the binary.
134 135 136 137 138 139 140 141

# Run unit tests.
npm test

# Serve the test site (visit https://localhost to view).
web-server test/site
```

142 143 144 145 146 147 148 149
__Note:__ for commands that require root privileges (i.e., `enable` and `disable`), Indie Web Server will automatically restart itself using sudo and Node must be available for the root account. If you’re using [nvm](https://github.com/creationix/nvm), you can enable this via:

```shell
# Replace v10.15.3 with the version of node you want to make available globally.
sudo ln -s "$NVM_DIR/versions/node/v10.15.3/bin/node" "/usr/local/bin/node"
sudo ln -s "$NVM_DIR/versions/node/v10.15.3/bin/npm" "/usr/local/bin/npm"
```

Aral Balkan's avatar
Aral Balkan committed
150 151 152 153 154 155 156 157 158 159 160
### Native binaries

```shell
# Clone and install.
git clone https://source.ind.ie/hypha/tools/web-server.git
cd web-server
npm i         # Install modules and development dependencies.

# Run unit tests.
npm test

161 162
# Build the native binary for your platform.
# To build for all platforms, use npm run build -- --all
Aral Balkan's avatar
Aral Balkan committed
163 164 165
npm run build

# Serve the test site (visit https://localhost to view).
166 167
# e.g., To run the version 9.2.2 Linux binary:
dist/linux/9.2.2/web-server test/site
168 169
```

170 171 172 173 174 175
### Build and install native binary locally

```shell
npm run install-locally
```

176 177 178 179 180 181 182
### Deployment

```shell
# To build binaries for both linux and macOS and also to
# copy them over to the Indie Web Site for deployment.
# (You will most likely not need to do this.)
npm run deploy
Aral Balkan's avatar
Aral Balkan committed
183 184
```

185
## Syntax
186

Aral Balkan's avatar
Aral Balkan committed
187
```shell
Aral Balkan's avatar
Aral Balkan committed
188
web-server [command] [folder|host] [host] [--options]
Aral Balkan's avatar
Aral Balkan committed
189
```
190

Aral Balkan's avatar
Aral Balkan committed
191 192 193
  * `command`: version | help | local | global | proxy | sync | enable | disable | logs | status
  * `folder|host`: Path of folder to serve (defaults to current folder) or host to proxy or sync.
  * `host`: Host to sync.
194
  * `options`: Settings that alter server characteristics.
Aral Balkan's avatar
Aral Balkan committed
195

196 197 198 199
### Commands:

  * `version`: Display version and exit.
  * `help`: Display help screen and exit.
200 201
  * `local`: Start server as regular process with locally-trusted certificates.
  * `global`: Start server as regular process with globally-trusted certificates.
Aral Balkan's avatar
Aral Balkan committed
202 203
  * `proxy`: Start server to proxy provided HTTP URL via HTTPS. Also proxies WebSockets.
  * `sync`: Start server as regular process with locally-trusted certificates and rsync folder to host.
Aral Balkan's avatar
Aral Balkan committed
204

205
On Linux distributions with systemd, you can also use:
Aral Balkan's avatar
Aral Balkan committed
206

207 208
  * `enable`: Start server as daemon with globally-trusted certificates and add to startup.
  * `disable`: Stop server daemon and remove from startup.
209
  * `logs`: Display and tail server logs.
210
  * `status`: Display detailed server information.
Aral Balkan's avatar
Aral Balkan committed
211

212
If `command` is omitted, behaviour defaults to `local`.
213 214

### Options:
Aral Balkan's avatar
Aral Balkan committed
215

Aral Balkan's avatar
Aral Balkan committed
216 217 218 219 220 221 222 223 224 225 226 227
  * `--port=N`: Port to start server on (defaults to 443).

#### For the enable command:

  * `--sync`: Ensure the server can also rsync via ssh (so you can sync your site to it from your local machine).

### For the sync command:

  * `--host`: The remote host to sync to (e.g., my-demo.site).
  * `--account`: The ssh account to use on remote server (defaults to same as on current session).
  * `--folder`:	The subfolder of home folder to sync to on remote machine (defaults to name of served folder).
  * `--proxy`: Proxy the specified host and port instead of starting a regular local server.
Aral Balkan's avatar
Aral Balkan committed
228

229
All command-line arguments are optional. By default, Indie Web Server will serve your current working folder over port 443 with locally-trusted certificates.
Aral Balkan's avatar
Aral Balkan committed
230

231
If you want to serve a directory that has the same name as a command, you can specify the command in _options_ format. e.g., `web-server --enable logs` will start Indie Web Server as a startup daemon to serve the _logs_ folder.
232

233
When you use the `global` or `enable` commands, globally-trusted Let’s Encrypt TLS certificates are automatically provisioned for you using ACME TLS the first time you hit your hostname. The hostname for the certificates is automatically set from the hostname of your system (and the _www._ subdomain is also automatically provisioned).
234

Aral Balkan's avatar
Aral Balkan committed
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266
## Usage examples

### Develop using locally-trusted certificates:

| Goal                                      | Command                                                       |
| ----------------------------------------- | ------------------------------------------------------------- |
| Serve current folder (shorthand)          | web-server                                                    |
| Serve folder site (shorthand)             | web-server site                                               |
| Serve current folder                      | web-server local                                              |
| Serve folder site                         | web-server local site                                         |
| Serve folder site at port 666             | web-server local site --port=666                              |
| Proxy localhost:1313 to https://localhost | web-server proxy localhost:1313                               |
| Serve current folder, sync it to my.site  | web-server sync my.site                                       |
| Serve site folder, sync it to my.site     | web-server sync site my.site                                  |
| Ditto, but using the --host option        | web-server sync site --host=my.site                           |
| Ditto, but use account me on my.site      | web-server sync site --host=my.site --account=me              |
| Ditto, but sync to remote folder www      | web-server sync site --host=my.site --account=me --folder=www |
| Ditto, but using the --to option          | web-server sync site --to=me@my-site:/home/me/www             |
| Sync current folder, proxy localhost:1313 | web-server sync my.site --proxy=localhost:1313                |

### Stage and deploy using globally-trusted Let’s Encrypt certificates:

| Goal                                      | Command                                                       |
| ----------------------------------------- | ------------------------------------------------------------- |
| Serve current folder                      | web-server global                                             |
| Serve folder site                         | web-server global site                                        |
| Serve current folder as daemon            | web-server enable                                             |
| Ditto & also ensure it can rsync via ssh  | web-server enable --sync                                      |
| Get status of deamon                      | web-server status                                             |
| Display server logs                       | web-server logs                                               |
| Stop current daemon                       | web-server disable                                            |

267
## Native support for an Evergreen Web
268

269 270
What if links never died? What if we never broke the Web? What if it didn’t involve any extra work? It’s possible. And, with Indie Web Server, it’s easy.

271
### Native cascading archives support
272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290

If you have a static archive of the previous version of your site, you can have Indie Web Server automatically serve it for you. For example, if your site is being served from the `my-site` folder, just put the archive of your site into a folder named `my-site-archive-1`:

```
|- my-site
|- my-site-archive-1
```

If a path cannot be found in `my-site`, it will be served from `my-site-archive-1`.

And you’re not limited to a single archive (and hence the “cascade” bit in the name of the feature). As you have multiple older versions of your site, just add them to new folders and increment the archive index in the name. e.g., `my-site-archive-2`, `my-site-archive-3`, etc.

Paths in `my-site` will override those in `my-site-archive-3` and those in `my-site-archive-3` will, similarly, override those in `my-site-archive-2` and so on.

What this means that your old links will never die but if you do replace them with never content in never versions, those will take precedence.

### Native 404 → 302 support

But what if the previous version of your site is a dynamic site and you either don’t want to lose the dynamic functionality or you simply cannot take a static backup. No worries. Just move it to a different subdomain or domain and make your 404s into 302s.
291

Aral Balkan's avatar
Aral Balkan committed
292
Indie Web Server has native support for [the 404 to 302 technique](https://4042302.org) to ensure an evergreen web. Just serve the old version of your site (e.g., your WordPress site, etc.) from a different subdomain and tell Indie Web Server to forward any unknown requests on your new static site to that subdomain so that all your existing links magically work.
293 294 295

To do so, create a simple file called `4042302` in the root directory of your web content and add the URL of the server that is hosting your older content. e.g.,

296
### /4042302
297
```
Aral Balkan's avatar
Aral Balkan committed
298
https://the-previous-version-of.my.site
299
```
300 301 302 303 304 305

You can chain the 404 → 302 method any number of times to ensure that none of your links ever break without expending any additional effort to migrate your content.

For more information and examples, see [4042302.org](https://4042302.org).

## Custom error pages
306

307 308
![Screenshot of the custom 404 error page included in the unit tests](images/custom-404.png)

309 310 311 312
You can specify a custom error page for 404 (not found) and 500 (internal server error) errors. To do so, create a folder with the status code you want off of the root of your web content (i.e., `/404` and/or `/500`) and place at least an `index.html` file in the folder. You can also, optionally, put any assets you want to display on your error pages into those folders and load them in via relative URLs. Your custom error pages will be served with the proper error code and at the URL that was being accessed.

If you do not create custom error pages, the built-in default error pages will be displayed for 404 and 500 errors.

313 314
When creating your own servers (see [API](#API)), you can generate the default error pages programmatically using the static methods `WebServer.default404ErrorPage()` and `WebServer.default500ErrorPage()`, passing in the missing path and the error message as the argument, respectively to get the HTML string of the error page returned.

315
## API
316

317
Indie Web Server’s `createServer` method behaves like the built-in _https_ module’s `createServer` function. Anywhere you use `require('https').createServer`, you can simply replace it with `require('@ind.ie/web-server').createServer`.
318

319

320
### createServer([options], [requestListener])
321

322
  - __options__ _(object)_: see [https.createServer](https://nodejs.org/api/https.html#https_https_createserver_options_requestlistener). Populates the `cert` and `key` properties from the automatically-created [nodecert](https://source.ind.ie/hypha/tools/nodecert/) or Let’s Encrypt certificates and will overwrite them if they exist in the options object you pass in. If your options has `options.global = true` set, globally-trusted TLS certificates are obtained from Let’s Encrypt using ACME TLS.
323

324
  - __requestListener__ _(function)_: see [https.createServer](https://nodejs.org/api/https.html#https_https_createserver_options_requestlistener). If you don’t pass a request listener, Indie Web Server will use its default one.
325

326
    __Returns:__ [https.Server](https://nodejs.org/api/https.html#https_class_https_server) instance, configured with either locally-trusted certificates via nodecert or globally-trusted ones from Let’s Encrypt.
327

328
#### Example
329 330

```js
Aral Balkan's avatar
Aral Balkan committed
331
const webServer = require('@ind.ie/web-server')
332 333 334 335 336
const express = require('express')

const app = express()
app.use(express.static('.'))

337
const options = {} // to use globally-trusted certificates instead, set this to {global: true}
Aral Balkan's avatar
Aral Balkan committed
338
const server = webServer.createServer(options, app).listen(443, () => {
339 340 341 342
  console.log(` 🎉 Serving on https://localhost\n`)
})
```

343
### serve([options])
344 345

Options is an optional parameter object that may contain the following properties, all optional:
346

347
  - __path__ _(string)_: the directory to serve using [Express](http://expressjs.com/).static.
348

349
  - __callback__ _(function)_: a function to be called when the server is ready. If you do not specify a callback, you can specify the port as the second argument.
350

351
  - __port__ _(number)_: the port to serve on. Defaults to 443. (On Linux, privileges to bind to the port are automatically obtained for you.)
352

353
  - __global__ _(boolean)_: if true, globally-trusted Let’s Encrypt certificates will be provisioned (if necesary) and used via ACME TLS. If false (default), locally-trusted certificates will be provisioned (if necesary) and used using nodecert.
354 355

    __Returns:__ [https.Server](https://nodejs.org/api/https.html#https_class_https_server) instance, configured with either locally or globally-trusted certificates.
356 357


358
#### Examples
359

Aral Balkan's avatar
Aral Balkan committed
360
Serve the current directory at https://localhost using locally-trusted TLS certificates:
361

362
```js
Aral Balkan's avatar
Aral Balkan committed
363 364
const webServer = require('@ind.ie/web-server')
const server = webServer.serve()
365 366
```

Aral Balkan's avatar
Aral Balkan committed
367
Serve the current directory at your hostname using globally-trusted Let’s Encrypt TLS certificates:
368 369

```js
Aral Balkan's avatar
Aral Balkan committed
370 371
const webServer = require('@ind.ie/web-server')
const server = webServer.serve({global: true})
372 373
```

374 375
## Contributing

Aral Balkan's avatar
Aral Balkan committed
376
Indie Web Server is, by design, a zero-configuration personal web server for single-tenant web applications for and by individuals. As such, any new feature requests will have to be both fit for purpose and survive a trial by fire to be considered. (That is, this is [Small Tech](https://ar.al/2019/03/04/small-technology/), with the emphasis on _small_).
377 378 379

Please file issues and submit pull requests on the [Indie Web Server Github Mirror](https://github.com/indie-mirror/indie-web-server).

380 381
## Help wanted

Aral Balkan's avatar
Aral Balkan committed
382 383 384
For locally-trusted certificates, all dependencies are installed automatically for you if they do not exist if you have apt, pacman, or yum (untested) on Linux or if you have [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/) (untested) on macOS.

I can use your help to test Indie Web Server on the following platform/package manager combinations:
385

386 387
  - Linux with yum
  - macOS with MacPorts
388

Aral Balkan's avatar
Aral Balkan committed
389
Please [let me know how/if it works](https://github.com/indie-mirror/web-server/issues). Thank you!
390 391 392 393

## Thanks

  * [thagoat](https://github.com/thagoat) for confirming that [installation works on Arch Linux with Pacman](https://github.com/indie-mirror/https-server/issues/1).
394

395
  * [Tim Knip](https://github.com/timknip) for confirming that [the module works with 64-bit Windows](https://github.com/indie-mirror/https-server/issues/2) with the following behaviour: “Install pops up a windows dialog to allow adding the cert.” __Note: Indie Web Server is not supported on Windows. Please use Windows Subsystem for Linux.__
396

397
  * [Run Rabbit Run](https://hackers.town/@nobody) for [the following information](https://hackers.town/@nobody/101670447262172957) on 64-bit Windows: “Win64: works with the windows cert install popup on server launch. Chrome and ie are ok with the site then. FF 65 still throws the cert warning even after restarting.” __Note: Indie Web Server is not supported on Windows. Please use Windows Subsystem for Linux.__