Behind the Scenes with Two New Salary Transparency Websites
đ Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one out of five topics in todayâs subscriber-only The Scoop issue. If youâre not yet a full subscriber, you missed this weekâs deep-dive into Figmaâs engineering culture. To get full newsletters twice a week, subscribe here.
In The Scoop #42, I went into detail about pay transparency changes in the US. California, Washington State and New York all recently passed legislation that mandates employers to post salary or total compensation ranges for job postings. This regulation impacts tech companies with a sizable presence in these regions, and is especially relevant for Big Tech. Companies have started to comply by listing expected pay upfront.
Starting January, a wealth of compensation data is available on these employersâ websites. This created an opportunity to build job sites which collect this data, make it easy to browse, and allow job seekers to apply to jobs paying at or above a certain level.
Several software engineers rose to the occasion by doing precisely this over the past few months. I reached out to them for some âbehind the scenesâ insights.
Comprehensive.io crawls job sites and parses the data. Users can browse by company and range:

The site was cofounded - and is partially built - by Roger Lee, the creator of job cuts tracker website Layoffs.fyi. The site was launched in January 2023, shortly after California's pay transparency law went live. Today, the site tracks 2,000 tech companies and startups. In what is neat, the site tracks what percentage of companies are in compliance with the law: that is, post salary ranges now that they are required to do so.
I asked Roger why he started the site. He said:
âAs a result of running Layoffs.fyi, Iâm also keenly aware that hundreds of thousands of tech employees have lost their jobs over the past year. Our hope is that making salary ranges more accessible on Comprehensive.io will help level the playing field for job seekers and employees, and help them navigate this fast-evolving talent market.â
How does Comprehensive.io work and what is the tech stack behind it? Roger explains:
âWe built software that automatically visits the Careers pages of these companies every day, finds all the job posts, and uses AI to extract the salary ranges from the text of the job descriptions. For AI, Â weâve built a system to efficiently use GPT-4 for this purpose, including auto-crafting prompts and performing pre and post-processing.
We use React on the frontend, Node.js on the backend, and Postgres for database storage.â
Neat - and itâs nice to hear of the team already adopting GPT-4 for a practical use case. Given Roger has been running the site for more than 3 months, I asked him what interesting things heâs come across. He mentioned 3 things:
â1. Although much has been made about outliers like Netflix who post unhelpfully-wide salary ranges, it turns out the vast majority of companies are posting salary ranges in good faith.
For example, the average salary range posted on a job listing is $130,000 - $200,000. This represents a width of +/- 21% from the midpoint: right in line with what HR professionals consider to be best practice for the width of an internal salary range.
2. The percentage of tech companies complying with the California transparency law has jumped from 28% on January 1âwhen the law went liveâ to 58% today. In NYCâwhere the law has been live since last Octoberâ, the compliance rate is 70%.
3. The average salary posted for a âMachine Learning Engineerâ job title is 19% higher than for a âSoftware Engineerâ job title!â
Finally, I asked Roger if he can share the highest and lowest compensation packages they picked up on. The highest ones are:
- Netflix advertises total compensation of $300,000 - $900,000
- Mashin advertises a salary range of $250,000 - $650,000
- OpenAI lists a salary range of $200,000 - $370,000
- Airtable lists a salary range of $248,000 - $322,905
And the lowest ones:
- Remesh lists a "minimum salary" of $100,000
- FourKites lists a salary range of $75,000 - $105,000
- Illumina lists a salary range of $79,300 - $118,900
Levels.fyi Jobs is similar to Comprehensive.io, except it also tries to fill the gap on total compensation estimates for job postings that list base salary, but not equity or bonus components:

I reached out to Levels.fyi cofounder Zuhayeer Musa, asking how the site works. He shared:
âI'd preface everything by saying that this is very much a v1 of our jobs product and we plan to iterate and build a lot more as we get feedback.
We use a data provider to get jobs data and donât scrape ourselves. Most jobs vendors have a ton of âjunk jobs,â so we spent a fair bit of time culling the dataset to jobs that are unique.
We put the jobs data into Amazon S3. We have a network of Lamdas that fire any time new data is added. Our system is using purely Serverless to process the data. During processing, we match companies, titles and more, with our dataset. We also enrich the jobs with total compensation estimates and benefits in this step.â
Itâs early days but I asked Zuhayeer what was an engineering challenge theyâve encountered. It was around unique searches:
âWe were surprised to see just how many job searches are unique! For the landing page, this is less of a problem as we use caching heavily. However, we wanted to build powerful filters for users. We have filters like the ability to filter by particular benefits, total comp, company valuation, and others.
However, all these filters mean that there are more unique requests, and we canât take advantage of caching as much as weâd like.â
Zuhayeer shared that theyâre playing around with OpenAI embeddings and vector databases:
âWeâre still experimenting with some better search techniques and have started prototyping semantic search powered by OpenAI embeddings. We are super excited to start playing vector databases - ones that store and index vector embeddings we get from natural language processing models like OpenAI embeddings.â
Finally, Zuhayeer asked me to relay that theyâd love to get feedback if you use the site - via the on-site widget or the hello@levels.fyi email.
Thanks both to Roger and Zuhayeer for sharing details, and browse their sites here:
This was one out of the five topics covered in this weekâs The Scoop. A lot of what I share in The Scoop is exclusive to this publication, meaning itâs not been covered in any other media outlet before and youâre the first to read about it.
The full The Scoop edition additionally covers:
- Big Tech realizes quick layoffs in Europe canât be done. Meta and Amazon are realizing what companies like Microsoft and Uber learned the hard way: that in Europe you cannot hire fast, then fire fast. I share my first-hand experience of Uberâs 2019 and 2020 layoffs in the Netherlands, and what employee protection might mean for Big Tech and hiring in Europe. Exclusive.
- Mercedes-Benzâs compensation philosophy. A deep dive into how software engineers at the German car maker are paid in Germany, and the influence the countryâs biggest workerâs union plays in salary bands. Exclusive.
- Unexpectedly big raises at Hubspot. Iâve talked with an engineering manager whoâs been surprised by how high compensation increases are at the publicly traded tech company. Exclusive.
- Total compensation drops at Amazon. This week is when engineers at Amazon learn their compensation rises. Engineers told me how things are going, and Iâve heard disappointment and a drop in total compensation. Exclusive.
Featured Pragmatic Engineer Jobs
- Full-Stack Engineer at Farmlend. ÂŁ85-95K + equity. London.
- Senior Backend Engineer at Farmlend. ÂŁ85-95K + equity. London.
- Senior Full Stack Engineer at Perfect Venue. $150-180K + equity. San Francsico or Remote (US).
- Senior Software Engineer at AI Build. London or Remote (Europe).
- Senior DevOps Engineerr at AI Build. Remote (US).
- Full-Stack Engineer at Vital. $70-120K + equity. Remote (Global, within 5 hours of GMT).
- Backend Engineer at Vital. $70-120K + equity. Remote (Global, within 5 hours of GMT).
- Technical Lead - Platform at Vannevar Labs. Remote (US).
- Senior Software Engineer, Fullstack at Vannevar Labs. Remote (US).
- Computational Geometry Engineer at AI Build. London or Remote (Europe).
- Senior QA Engineer at AI Build. London or Remote (Europe).
- Lead Backend Developer at Cineville. âŹ53-79K + equity. Amsterdam.
- Senior Software Engineer, Distributed Systems at Mixpanel. $200-270K + equity. New York, San Franciso, Seattle or Remote (US).
- Senior Software Engineer, Fullstack at Mixpanel. $200-270K + equity. New York, San Franciso, Seattle or Remote (US).
- Principal Engineer at Shoplift. $185-205K. New York.
- Senior Engineer at Sixfold AI. New York.
The above jobs score at least 10/12 on The Pragmatic Engineer Test. Browse more senior engineer and engineering leadership roles with great engineering cultures, or add your own on The Pragmatic Engineer Job board and apply to join The Pragmatic Engineer Talent Collective.
Want to get interesting opportunities from vetted tech companies? Sign up to The Pragmatic Engineer Talent Collective and get sent great opportunities - similar to the ones below without any obligation. You can be public or anonymous, and Iâll be curating the list of companies and people.
Are you hiring senior+ engineers or engineering managers? Apply to join The Pragmatic Engineer Talent Collective to contact world-class senior and above engineers and engineering managers/directors. Get vetted drops twice a month, from software engineers - full-stack, backend, mobile, frontend, data, ML - and managers currently working at Big Tech, high-growth startups, and places with strong engineering cultures. Apply here.