This page reads a slice of an Apache or nginx access log and summarises what is in it: how many requests were parsed, how many distinct IPs made them, how much of the traffic came from crawlers, which paths were requested most and where the 4xx and 5xx responses are. Everything runs in the browser tab.
It is a quick second opinion, not a replacement for a full log analyser. A pasted slice is useful for a specific question, such as whether Googlebot is hitting a dead page or whether a bot is spending its budget on parameter URLs. It is not a crawl-budget report for the whole month.
What it can read
The parser expects one request per line in common or combined log format. A combined line has the client IP, the ident and user fields, a timestamp in brackets, the request in quotes, the status code, the response size, then the referer and user agent in quotes. The common format drops the last two fields, and the parser still accepts it.
Query strings are kept, so /page and /page?utm_source=newsletter are counted as separate paths. That is useful when you want to see parameter variants, and it also splits what a crawler may treat as one page. The path list can show between 3 and 40 rows; the default is 10.
The first 20,000 lines are parsed. Anything beyond that is ignored, so a multi-day log needs to be cut down before you paste it. JSON logs, W3C extended logs and custom formats are not understood, and their lines are counted as unparsed rather than guessed at.
A worked example
Twelve lines were pasted: eleven valid combined-format requests and one line of prose. The summary read 11 requests from 7 IPs, 54.5% bot. Six of the eleven requests carried a recognisable crawler user agent, and four ended in a 4xx or 5xx status.
| Path | Hits | Errors | Share |
|---|---|---|---|
| /missing-page | 2 | 2 | 18.2% |
| /tools/pdf-tools/merge-pdf.html | 2 | 0 | 18.2% |
| /blog/ | 1 | 1 | 9.1% |
| /tools/calculator/percentage-calculator.html | 1 | 1 | 9.1% |
| /tools/seo-tools/keyword-density-checker.html?utm_source=newsletter | 1 | 0 | 9.1% |
Two paths tied at two hits. The tool put /missing-page first because it breaks ties alphabetically, and both were 18.2% of the slice. The bot breakdown was AhrefsBot 2, Googlebot 2, Bingbot 1 and GPTBot 1. The four errors sat on three URLs: two 404s on /missing-page, one 404 on /blog/ and one 500 on a calculator page.
That is the useful pattern the page is designed to surface. A dead URL taking the joint-largest share of requests, requested twice by a crawler, is a crawl-budget leak you can fix with a redirect. The 500 on a live page is a server problem, not an SEO one, and it is worth separating from the 404s.
How bots are counted
The tool matches the user agent against a short list: Googlebot, Bingbot, GPTBot, ClaudeBot, AhrefsBot, SemrushBot and Bytespider. Anything else containing bot, crawler, spider, slurp, curl, wget, python-requests or headless is grouped as Other bot. A request with a normal browser string is counted as human.
This is a label from a string the client chooses. It is not verification. Anyone can send a Googlebot user agent, and a real crawler can occasionally send a different one. The page says so in the note, and the Separate bot and human requests checkbox does less than its label suggests: the two totals are always reported, and ticking the box only adds a sentence to the note.
If the distinction matters for a decision, verify the IP ranges or use a tool that does reverse DNS checks. This one deliberately keeps the request local and does no network lookup.
Reading errors and crawl waste
An error rate above 5% turns the checklist amber and above 15% turns it red. Those bands are editorial, not Google thresholds. What matters is which URLs are failing and who is asking for them. A 404 requested once by a person is a broken link; the same 404 requested repeatedly by a crawler is a page that needs to exist or redirect.
The error list shows the path, the status, the number of hits and, when the request was a bot, a second number. That second number is a count, not a name: the line reads 404, 1 hit, 1 rather than 1 Googlebot hit. It is a display bug, and the crawler table above is the place to see which bot was involved.
One grammar defect appears when a line is skipped: the warning reads 1 line was skipped because they did not match the combined log format. The count and the reason are right; the pronoun is not.
Turning the slice into a fix
Start with the paths that combine errors and bot traffic. Redirect dead URLs that still receive requests, fix or remove internal links to them, and check whether a query-string variant is being crawled separately from the canonical page. If a parameter is only for tracking, the robots.txt generator can block the pattern, though blocking a crawl does not remove a URL from the index.
Then check the sitemap. A URL that appears in the XML sitemap but returns a 404 is a signal you can fix at the source. A URL that is requested but missing from the sitemap may be an orphan worth adding, or a page worth removing.
Finally, keep the slice honest. Paste a full day or a full week rather than a handful of lines, note the date range beside the numbers, and do not compare a mobile-heavy slice with a desktop one. The page reports what you gave it and nothing more.