indiecheck

# indiechecker

Point it at a web page and it lists every microformat, rel link, IndieWeb endpoint and meta tag it finds there, then tells you what to fix, with the HTML to fix it.


## What it reads

**microformats2.** The vocabularies on the wiki, stable and draft: h-card, h-entry, h-feed, h-event, h-cite, h-adr, h-geo, h-product, h-recipe, h-review, h-review-aggregate, h-resume, h-item, h-listing, h-measure, h-app, h-breadcrumb, h-org and h-food. Anything under `h-x-` shows up as experimental. Types nobody has heard of still get parsed, with a note saying so.

**Classic microformats.** The pre-2010 class names: vcard, adr, geo, hentry, hfeed, hnews, vevent, vcalendar, hreview, hreview-aggregate, hrecipe, hproduct, hresume, hlisting, hmedia, haudio and xoxo. Plus rel-tag, rel-license, rel-nofollow, XFN and VoteLinks. Items that came from classic markup are marked with the class they came from.

**IndieWeb.** The representative h-card, who wrote each h-entry (following the authorship rules: the entry, then the parent h-feed, then rel=author), rel=me, Webmention, Micropub, Microsub, IndieAuth and WebSub. Endpoints are read from the HTML and from the HTTP `Link` header, which is where some sites keep them.

**Meta tags.** Title, description, lang, charset, viewport, canonical, Open Graph, Twitter cards, `fediverse:creator`, theme-color, icons, the manifest and feeds. Declared feeds get requested, so a feed that answers 401 or comes back as `text/plain` gets flagged.

**The site around the page.** robots.txt (missing, or blocking every crawler), the sitemaps it declares or the one at `/sitemap.xml`, whether the page ends up on https, whether rel=canonical points to another host, and whether both `www.` and the bare domain serve the site without one redirecting to the other. These checks only run on a fetched URL, not on pasted HTML.

**Advice.** Errors break parsing, like a `dt-published` that isn't a date or a class called `hcard`. Warnings leave out something tools look for, like an h-entry without an author. Tips are nice to have. Each one says where the problem is and comes with a snippet. When the page's `generator` meta tag says WordPress, Hugo or Eleventy, the snippets for a missing h-card, rel=me, feed or Webmention endpoint say where the code goes in that tool, or which plugin does it for you.

## Running it

You need PHP 8.2 or newer with the curl, dom and mbstring extensions, and Composer.

```bash
composer install
php -S 127.0.0.1:8123 -t public
```

Then open http://127.0.0.1:8123.

To put it online, point the web server's document root at `public/`. Nothing outside that folder should be reachable from the web.

## JSON API

Add `format=json` to any check:

```bash
curl "http://127.0.0.1:8123/?url=https://tantek.com/&format=json"
```

Or POST the HTML itself, with an optional `base` URL for resolving relative links:

```bash
curl -X POST http://127.0.0.1:8123/ \
  --data-urlencode "html@page.html" \
  -d "base=https://example.com/post" \
  -d "format=json"
```

The canonical mf2 parse is under `mf2`. The rest of the response is what the page shows: `summary`, `indieweb`, `rels`, `metadata`, `feed-checks`, `robots`, `domain` and `advice`. `robots` and `domain` are `null` for pasted HTML. A failed fetch answers 422 with `{"error": "..."}`.

## Markdown

`format=markdown` returns the same report as Markdown, served as `text/markdown`: the fetch facts, the counts, a short summary, then every finding grouped into errors, warnings and tips, with its fix in a code block. It's meant for pasting into an issue or keeping next to the site's source.

```bash
curl "http://127.0.0.1:8123/?url=https://tantek.com/&format=markdown" > report.md
```

## In CI

Add `fail=errors` and the response comes back as HTTP 412 when the page has any error. `fail=warnings` does the same for errors or warnings. It works with every format, so `curl -f` is enough to break a build:

```bash
curl -sf "https://indie.pablomurad.com/?url=https://example.com/&fail=errors" -o /dev/null
```

For finer rules, read the counts from the JSON:

```bash
curl -s "https://indie.pablomurad.com/?url=https://example.com/&format=json" \
  | jq -e '.summary.errors == 0 and .summary.warnings <= 2'
```

## Fetching

The checker requests pages from the server it runs on, so it's careful about where it goes:

- only http and https, on ports 80, 443, 8080 and 8443
- no private, loopback, link-local or otherwise reserved addresses, checked again after every redirect, and cURL is pinned to the IP that passed the check
- at most 5 redirects, 2 MB and 20 seconds per page
- up to 4 feeds per page, with 8 seconds each (only the headers are read)
- robots.txt at 64 KB and 8 seconds, then up to 2 of the sitemaps it declares, or `/sitemap.xml` when it declares none (headers only)
- one request to the `www.` or bare-domain twin of the host, when there is one (headers only)

That's at most 10 requests for one check.

A site on your own network can't be fetched. Paste its HTML instead.

## Where things are

- `public/index.php` takes the request and answers with a page, JSON or Markdown
- `src/Fetcher.php` does the HTTP requests and the address checks
- `src/Report.php` runs a check, including the robots.txt, sitemap and domain requests
- `src/Robots.php` reads robots.txt
- `src/Markdown.php` writes the Markdown report
- `src/Mf2Parser.php` extends php-mf2 so that top-level `geo`, `hreview-aggregate` and `hnews` get parsed, and marks items that came from classic markup
- `src/Microformats.php`, `src/Rels.php` and `src/Metadata.php` read the page
- `src/Advice.php` has every rule behind the advice list
- `src/Vocabulary.php` lists the vocabularies, their properties and the known rel values
- `views/` holds the PHP templates

## Credits

Parsing by [php-mf2](https://github.com/microformats/php-mf2). The layout follows the [Minimal](https://github.com/orderedlist/minimal) GitHub Pages theme by Steve Smith, rewritten from scratch with darker link and label colors so the text passes contrast checks. Type is [Noto Sans](https://fonts.google.com/noto/specimen/Noto+Sans) and [IBM Plex Mono](https://github.com/IBM/plex), both under the SIL Open Font License and served from `public/fonts/`.