Part 5 · 1 chapters · ~8 min
Evaluating Sources and Research
Where claims come from (vendor blogs, benchmarks, papers, conference talks, social media), checking benchmarks and their conditions, reading a paper efficiently, replication and publication bias, and building an evidence habit for technical decisions.
11
Grading the evidence
| source | typical weakness | what to check |
|---|---|---|
| vendor benchmark | chosen to flatter the product | hardware, configuration, workload; can you reproduce it? |
| engineering blog post | one company's context, survivorship | their scale and constraints versus yours |
| conference talk | the polished story, not the failures | what they would do differently |
| academic paper | idealised conditions, may not replicate | assumptions, evaluation setup, follow-up work |
| a thread or post | confidence without evidence | primary sources behind the claim |
| your own measurement | measurement mistakes (Computers P10) | method, variance, repeated runs |
Reading a paper in 20 minutes: abstract, then figures and tables, then the conclusion, then the evaluation setup; read the full text only if it still matters. Papers We Love and the Distributed Systems course part 9 are good starting lists.