14.9 KB
Newer Older
Aral Balkan's avatar
Aral Balkan committed
# Better
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
Better protects you from unethical web sites. It makes your web experience safer, lighter, and faster.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
Better enforces the [Ethical Design Manifesto]( It helps the Web respect human rights, effort, and experience.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
Better is curated by, a social enterprise that defends human rights. It’s free, open, and transparent.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
## Content
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
This repository contains the Better content: Better’s database of information on trackers and other malware as well as the web sites that host them.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
This content is in Blockdown format. Blockdown is an extension of Markdown with special vocabulary to describe web malware. Blockdown can also contain WebKit content blocking rules. The Blockdown pages in Better’s content repository both describe web malware and contain the rules to block them.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
This content is processed by [Better Builder]( to generate the [Better web site]( as well as the data for the [Better iOS App](, including a WebKit `blockerList.json` file.
Aral Balkan's avatar
Aral Balkan committed

Aral Balkan's avatar
Aral Balkan committed
A seminal advantage of Better is that its database is human-readable, open, and extensible via pull requests. (The database is curated by using the Ethical Design Manifesto as the criteria for blocking rules.)
Aral Balkan's avatar
Aral Balkan committed

Contributing to the content is as easy as creating an account on []( and editing a content page in your browser.

Aral Balkan's avatar
Aral Balkan committed
## I’m not a developer, I just want to experience a Better web.

[Get Better from the App Store.](

Aral Balkan's avatar
Aral Balkan committed
25 26
## How can I support Better?

Laura Kalbag's avatar
Laura Kalbag committed
Buying [Better on the App Store]( is one way to support us. If you want to help with the ongoing costs of developing and maintaining Better, you can [donate to]( or, even better, [become a patron]( by setting up a recurring donation.
Aral Balkan's avatar
Aral Balkan committed
28 29 30

## I’m a developer, let me in!

31 32
The easiest way to get started is to follow the instructions in the readme for the [Better iOS app]( repository.

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69
## Testing locally.

[Better Builder]( will automatically pick up your changes as you save and rebuild your local data.

To persist your changes locally, commit them in Git and push to origin:

git commit -am "My awesome content update"
git push origin master

Note that these changes will be destroyed if you run the Better Builder installer (or the Better iOS installer, which runs the Better Builder installer as part of its own installation process). To not lose any work, save your changes regularly by pushing to production, as explained below.

## Saving your changes by pushing to production

You can push to production with:


Or, manually run what the save script does, which is:

git push live master

## Deployment

If you have commit rights to the content repository, just run the deployment script:


This will create a tag (you will have to enter a tag mesage when prompted, describing the release) and push it to production. Please make sure that you have already committed your changes and pushed them to production either via `git push live master` or by running the `./save` script, which does the same thing.

# Guide to Blockdown

72 73
Better content is authored in Blockdown.

74 75
Blockdown is Markdown with an extended high-level vocabulary for describing web malware for the Better database.

## Sites
77 78 79

Site pages have the following sections:

### Ethical design violations
81 82 83 84 85

## Ethical design violations

This is a list of ethical design violations that gets converted to a collection of badges on the rendered site pages. The Trackers part of the list, detailed below, is updated automatically by [Better Inspector](
87 88 89

#### Trackers

90 91 92
The first badge is always the trackers badge. In Blockdown it is represented by a list item introduced by the word `Trackers`:

93 94 95 96 97 98
  * Trackers
    * Automatically
    * Generated
    * List
    * of
    * Trackers
99 100 101 102

This gets automatically translated by [Better Builder]( to a badge similar to the one below:

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the trackers badge](images/readme/better/trackers-badge-example.png)
104 105 106

Tapping on the badge displays a popover with links to the actual trackers.

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the trackers popover](images/readme/better/trackers-popover-example.png)

The other badges are manually added if they apply to the site in question:
110 111 112 113

#### Aggressive

* Aggressive
115 116

117 118
Attempts to block content blockers.

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the Aggressive Badge](images/readme/better/aggressive-badge-example.png)
120 121 122 123 124

#### Doorslam

* Doorslam
126 127

128 129
Interrupts and blocks using modal dialogs.

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the Doorslam Badge](images/readme/better/doorslam-badge-example.png)
131 132 133 134 135

#### Clickbait

* Clickbait
137 138

139 140
Uses exploitative, addictive content syndication network(s).

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the Clickbait Badge](images/readme/better/clickbait-badge-example.png)
142 143 144 145 146

#### Fingerprint

* Fingerprint
148 149

150 151
Uses hidden Canvas fingerprinting.

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the Fingerprint Badge](images/readme/better/fingerprint-badge-example.png)
153 154

#### Web Bug
156 157

* Web bug
159 160

161 162
Uses invisible tracking pixels.

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the Web Bugs Badge](images/readme/better/web-bugs-badge-example.png)
164 165

We might create new badges as and when we find new types of web malware and unethical practices to document and warn people about.
166 167 168 169 170 171 172

## After Better section

## After Better

The After Better section provides statistics about the before (without the Better content blocker active) and after (with the Better content blocker active) performance of a site.

It is automatically generated by [Better Inspector](

Laura Kalbag's avatar
Laura Kalbag committed
![Screenshot of the After Better Section](images/readme/better/after-better.png)
178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201

## Block Rules section

This is the section where we enter the actual WebKit content blocking rules. Each rule is written in a strict subset of MSON (Markdown JSON) and has a brief explanation detailing what the rule does and why.

The blocking rules in this section serve the following purposes, in line with the [Ethical Design Manifesto](

  * Remove any first-party trackers (respect human rights)
  * Improve the usability of the site by removing first-party impediments like doorslams (respect human effort)
  * Improve the experience of the site (respect human effort) – we should especially aim to create a better experience after trackers have been removed (like removing empty spaces left over, etc.)

Please note that this is not the place to put blocking rules for trackers. Each tracker encountered should be entered into the [Trackers](#trackers) section and you have its own page in the `/trackers` section of the content.

Here is an example of a site-specific blocking rule in MSON format:

- trigger:
  - url-filter:
- action:
  - type: block

202 203
The Blockdown parser in Better supports all of the [WebKit content blocking rules]( Instead of JSON, however, we enter blocking rules in MSON. All Blockdown rules are combined by Better Builder into a single `blockerList.json` file.

204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249
Blockdown differs from plain WebKit content blocker rules in several ways to make authoring easier and to aid in readability:

1. The default load type in Blockdown is third-party.
2. The default for rules to be case sensitive.

So, if you do not specify a `load-type` or `url-filter-is-case-sensitive` properties in your rules, they will behave as if you had specified:

- trigger:
  - url-filter: …
  - load-type: third-party
  - url-filter-is-case-sensitive: true
- action
  - …

You may, of course, override these by explicitly specifying those properties in your rules.

## Automatic URL filter compilation 

Blockdown automatically compiles simple `url-filter` properties to regular expressions with higher specificity as recommended in the [domain targeting recommendations by WebKit engineer Benjamin Poulain](

This means that you can author your entries in plain text, like this:


  - url-filter: some-domain.ext


And Blockdown will compile them into the following form in the blockerList.json:


  "url-filter": "^[^:]+://+([^:/]+\\.)?some-domain\\.ext[:/]?"


## Further reading on WebKit content blocking

  * [Introduction to WebKit Content Blockers](
  * [Targeting Domains with Content Blockers](
  * [Official Safari content-blocking rules documentation from Apple](

250 251 252 253
# Investigation process

Currently, you need to have commit rights to the Content repository to use the Better commandline commands. However, you can use Git directly to fork the repository and submit merge requests and you can [add and edit pages through the online GitLab interface]( without commit rights.

254 255
## Find who owns and runs the tracker

256 257 258 259 260 261 262
1. **Start by editing the tracker**

	better/edit drafts/trackers/

	This will create an issue in GitLab (or update an existing issue, if one already exists) and create or checkout a branch for you. It will also open your working copy of the tracker page in your system editor and in the browser.

2. **Enter the tracker URL into your browser in a private window to see if it loads.**

	Make sure you don’t have an VPNs or extensions blocking or making your browser behave differently from the norm. If you have any tracker blockers already enabled, it may make it harder to investigate!

3. **If it doesn’t load, or if you get a blank page, perform a whois.**

	We are currently using for these so we can link to is as a source when stating ownership information. However, you will sometimes get more information from a direct whois look-up on your machine. In Terminal: `whois`

4. **Some trackers use a domain proxy or a cloaking service** (e.g., Domains by Proxy) to further hide their origins. In this case:

	* Open up the source of a site that the tracker originated on in the Web Developer console (Timeline view) of Safari (or in the web inspector of your browser of choice)

276 277 278 279 280 281 282 283 284
	* Try to recreate the original call. This might give you more clues about its origin. 

To find which sites a tracker is on, perform a search on the ~/ folder. For example, you can open up the folder in your text editor and do a global search for the tracker name.

You can also use [Better Inspector]( to search for strings within requests. e.g., to find all URLs that contain **, run:

	./inquiry --local
285 286 287

Other useful tools:

* [Mozilla Lightbeam](

290 291
## Add the tracker/site name to the tracker markdown file

292 293
The name should be formatted as:

294 295 296 297
**TrackerName** by Corporation (domain.tld)

If the tracker name is the same as the corporation name *(e.g. Adlucent by Adlucent)* then just keep the tracker name, and don’t incude the corporation name.
299 300
*When you edit a tracker markdown file for the first time, the domain.tld will already be in the title.*

301 302
## Add the site description

Add a concise one-line description of what the tracker, or the tracker owner, does.
304 305 306

*Usually the tracker sites have vague marketing spiel to describe themselves. Often a clearer description can be found in their privacy policy. If you can’t find a concise description in their own words, try to find their entry on [Wikipedia](, Bloomberg or Crunchbase.*

307 308 309 310 311 312 313 314 315 316 317 318 319 320 321
Other useful tools:

  * [Wikipedia](

## Include references in Notes

* Whether it’s the domain whois, or where you found the site description, include a link back to every source in the Notes section.
* Include a link to the tracker/corporation Privacy Policy (if it exists!)
* If you end up looking through the source file to find more information, you can include relevant code snippets in markdown.

*You can use sub-lists in Notes by using indented lists in markdown.*
*[See the Demandbase tracker for an varied use of Notes](*

# Handling duplicate trackers

Loads of trackers have multiple domains for the same tracker, or group of trackers. In this case, we don’t want duplicate entries that don’t stay in sync.

1. The first tracker found and investigated is the canonical tracker.

326 327 328 329 330
2. Any further trackers with the same name/owner should link to the canonical tracker in place of the description. *Example from [ tracker](*

	> See [](/trackers/

332 333 334 335 336
3. The Ethical Design Violations are still necessary, as the type of violation might vary between the domains.

4. The Block Rule is still necessary, as it blocks this specific domain.

5. The only Notes necessary is the source for the domain origin. Any other notes can be added to the canonical tracker.

338 339
# Content authoring guidelines

340 341 342 343 344
	* Be brief: do not quote the whole privacy policy; pick out interesting bits.

	* You can editorialise (with restraint). Sometimes you just have to laugh at the ridiculousness of some of the trackers that we’re covering. It also helps, when trudging through the cesspit of surveillance capitalism to retain our humour. And it also makes the pages more interesting to read (we don’t want to create a dry database). Please only add editorial comments for something unusually important or to highlight egregious abuses. A good rule of thumb would be: “would this make a good slide in a presentation to illustrate the problem with this particular thing or practice?” Editorial comments should be brief, marked with ‘– Ed.’ and limited to at most one per tracker.

	* Use images (sparingly). Not every humdrum tracker page needs images. However, if you are making an editorial comment and you feel that a visual aid is important in highlighting the point, please feel free to use images. Images and screenshots should be 1,160px wide (to display well at their 580pt width on high resolution screens). Please resize and compress images properly. On Mac, [ImageOptim]( is a great application for compressing PNGs and [PhotoBulk]( is a convenient app for converting between formats.