CarAtlass is a Web Scraping API designed to collect car listings and vehicle data from websites like Copart, among others.
This is my first attempt at building a web scraping tool to gather real-time data from the web — without relying on paid APIs.
The project leverages Bright Data's Browser APIs for efficient and reliable scraping.
in terminal
git clone https://github.com/Sh0utaa/CarAtlassExpressJs.git
cd CarAtlassExpressJs
First of all this is an ExpressJs project so you must have NPM installed after that, you need to create a .env file in the root directory. This file is used to store environment-specific variables and is not included in version control for security reasons.
DATABASE_URL– The connection string used by Prisma to connect to your Supabase PostgreSQL database.DIRECT_URL– Used by Prisma CLI for internal operations such as migrations and schema introspection. It also connects to the same Supabase database.
AUTH– Your Bright Data authentication token or credentials. This is required to access their scraping infrastructure.SBR_WS_ENDPOINT– The WebSocket endpoint provided by Bright Data for initiating browser-based scraping sessions.
Make sure these variables are correctly set in your .env file before starting or developing the application.
DATABASE_URL=postgresql://your_user:your_password@your_host:your_port/your_database DIRECT_URL=postgresql://your_user:your_password@your_host:your_port/your_database
AUTH=your_bright_data_auth_token SBR_WS_ENDPOINT=wss://your-brightdata-websocket-endpoint
After setting up your .env file, you can install dependencies and start the server.
Use the following commands in your terminal:
npm install
npm run dev
Once the server is running, you can access the API via the following endpoint:
GET http://localhost:3000/api/car/search?query=your%20query
replace "Your query" with the car you want to scrape and enjoy the data :)
for more details about the project please open the overivew.txt or overview.docx file