Developing a Laravel GitHub Remote Operation Utility Class from Scratch

In Laravel projects, syncing remote resources (e.g., location data, configuration files, templates) from GitHub is a common requirement. This guide details building a robust GithubLocationHandler utility class that handles the full workflow of "remote resource query → download → unzip → validation," optimized for global networks and enterprise use cases.

1. Requirement Analysis & Architecture Design

Core Requirements

Technology Stack & Architecture

• Remote Requests: Laravel Http facade (timeout control, retries)

• Caching: Laravel Cache (Redis/file cache, 24-hour TTL default)

• File Operations: Laravel File facade + Zipper class (ZIP handling)

• Error Handling: Custom business exceptions + Laravel exception system

• Configuration: Laravel config files (environment variable injection)

Class Structure

class GithubLocationHandler
{
    protected string $repositoryUrl;
    protected string $remoteTreeApi;
    protected string $cacheKey;
    protected array $httpConfig;
    protected string $tempDir;
​
    public function __construct(array $config) {}
    public function getAvailableCountries(): array {} // Get available country codes
    protected function downloadZip(string $countryCode): string {} // Download ZIP package
    protected function extractZip(string $zipPath, string $countryCode): string {} // Extract ZIP
    public function syncCountryData(string $countryCode): array {} // Full sync workflow
    protected function validateDataDir(string $dataPath): bool {} // Validate data integrity
    protected function cleanTempFiles(string $zipPath, bool $keepDir = false): void {} // Cleanup
}

2. Full Implementation

Step 1: Initialize Configuration

Create config/plugins/location.php for flexible settings:

return [
    'github' => [
        'repo_url' => env('GITHUB_LOCATION_REPO', 'https://github.com/tekintian/tcms-locations'),
        'tree_api' => 'https://api.github.com/repos/tcms/locations/git/trees/master',
        'cache_expire' => 86400, // 24 hours
        'http' => [
            'timeout' => 30,
            'retry' => 2,
            'proxy' => env('HTTP_PROXY') // Optional proxy for global access
        ],
        'temp_dir' => storage_path('app/location-temp'),
    ]
];

Step 2: Core Function Development

(1) Fetch Available Countries

public function getAvailableCountries(): array
{
    $cacheKey = 'github:location:available_countries';
    return Cache::remember($cacheKey, $this->cacheExpire, function () {
        try {
            $response = Http::withoutVerifying()
                ->timeout($this->httpConfig['timeout'])
                ->retry($this->httpConfig['retry'], 1000)
                ->asJson()
                ->get($this->remoteTreeApi);
​
            if ($response->failed()) {
                logger()->warning('GitHub API request failed, using default country list');
                return ['cn', 'us', 'vn', 'jp', 'gb', 'de']; // Global default list
            }
​
            $treeData = $response->json('tree');
            $ignorePaths = ['.gitignore', 'README.md', 'LICENSE'];
            return array_filter(array_map(function ($item) use ($ignorePaths) {
                return ($item['type'] === 'tree' && !in_array($item['path'], $ignorePaths))
                    ? strtolower($item['path'])
                    : null;
            }, $treeData));
        } catch (\Throwable $e) {
            logger()->error('Country list fetch failed', ['error' => $e->getMessage()]);
            return ['cn', 'us', 'vn', 'jp', 'gb', 'de'];
        }
    });
}

(2) Download ZIP Package

protected function downloadZip(string $countryCode): string
{
    $availableCountries = $this->getAvailableCountries();
    if (!in_array(strtolower($countryCode), $availableCountries)) {
        throw new RemoteLocationException("Country data unavailable (code: $countryCode)", 404);
    }
​
    $zipUrl = "{$this->repositoryUrl}/archive/refs/heads/master.zip";
    $zipPath = "{$this->tempDir}/{$countryCode}-location.zip";
​
    // Avoid duplicate downloads
    if (File::exists($zipPath) && File::size($zipPath) > 1024) {
        return $zipPath;
    }
​
    try {
        $response = Http::withoutVerifying()
            ->timeout($this->httpConfig['timeout'])
            ->retry($this->httpConfig['retry'], 1000)
            ->withOptions(['proxy' => $this->httpConfig['proxy'] ?? null])
            ->sink($zipPath)
            ->get($zipUrl);
​
        if (!$response->ok() || File::size($zipPath) <= 1024) {
            File::delete($zipPath);
            throw new \Exception("Download failed (status: {$response->status()})");
        }
        return $zipPath;
    } catch (\Throwable $e) {
        throw new RemoteLocationException("ZIP download failed: {$e->getMessage()}", 500);
    }
}

(3) Extract & Validate ZIP

protected function extractZip(string $zipPath, string $countryCode): string
{
    $extractDir = "{$this->tempDir}/{$countryCode}";
    $requiredFiles = ['country.json', 'states.json', 'cities.json'];
​
    // Clean existing directory
    if (File::isDirectory($extractDir)) {
        File::deleteDirectory($extractDir);
    }
​
    try {
        $zipper = new Zipper();
        $zipper->open($zipPath)->extractTo($extractDir);
        $zipper->close();
​
        // Locate actual data directory (handle repo prefix)
        $realDataDir = $this->findDataDir($extractDir, $countryCode);
        if (!$realDataDir) {
            throw new \Exception("Country directory not found: $countryCode");
        }
​
        // Validate core files
        foreach ($requiredFiles as $file) {
            if (!File::exists("{$realDataDir}/{$file}")) {
                throw new \Exception("Missing core file: $file");
            }
        }
        return $realDataDir;
    } catch (\Throwable $e) {
        if (File::isDirectory($extractDir)) {
            File::deleteDirectory($extractDir);
        }
        throw new RemoteLocationException("ZIP extraction failed: {$e->getMessage()}", 500);
    }
}
​
protected function findDataDir(string $baseDir, string $countryCode): ?string
{
    if (File::isDirectory("{$baseDir}/{$countryCode}")) {
        return "{$baseDir}/{$countryCode}";
    }
    foreach (File::directories($baseDir) as $subDir) {
        if (File::isDirectory("{$subDir}/{$countryCode}")) {
            return "{$subDir}/{$countryCode}";
        }
    }
    return null;
}

(4) Full Sync Workflow

public function syncCountryData(string $countryCode, bool $keepTemp = false): array
{
    $countryCode = strtolower($countryCode);
    try {
        $zipPath = $this->downloadZip($countryCode);
        $dataDir = $this->extractZip($zipPath, $countryCode);
        
        if (!$keepTemp) {
            $this->cleanTempFiles($zipPath);
        }
​
        return [
            'error' => false,
            'message' => "Sync successful (country code: $countryCode)",
            'data_dir' => $dataDir,
            'country_code' => $countryCode
        ];
    } catch (RemoteLocationException $e) {
        return [
            'error' => true,
            'message' => $e->getMessage(),
            'error_code' => $e->getErrorCode(),
            'country_code' => $countryCode
        ];
    }
}

3. Integration & Testing

Controller Integration

namespace App\Http\Controllers\Api;
use App\Http\Controllers\Controller;
use App\Services\Location\GithubLocationHandler;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Validator;
​
class LocationController extends Controller
{
    protected GithubLocationHandler $handler;
​
    public function __construct(GithubLocationHandler $handler)
    {
        $this->handler = $handler;
    }
​
    // Sync country data API
    public function syncCountryData(Request $request)
    {
        $validator = Validator::make($request->all(), [
            'country_code' => 'required|string|size:2|alpha',
            'keep_temp' => 'boolean',
        ], [
            'country_code.required' => 'Country code is required',
            'country_code.size' => 'Country code must be 2 letters (e.g., cn, us)',
        ]);
​
        if ($validator->fails()) {
            return response()->json([
                'code' => 400,
                'message' => $validator->errors()->first(),
                'data' => []
            ], 400);
        }
​
        $countryCode = strtolower($request->input('country_code'));
        $keepTemp = $request->input('keep_temp', false);
        $result = $this->handler->syncCountryData($countryCode, $keepTemp);
​
        if ($result['error']) {
            return response()->json([
                'code' => $result['error_code'],
                'message' => $result['message'],
                'data' => ['country_code' => $countryCode]
            ], $result['error_code']);
        }
​
        // Import data to database (example)
        $this->importDataToDatabase($result['data_dir'], $countryCode);
​
        return response()->json([
            'code' => 200,
            'message' => $result['message'],
            'data' => [
                'country_code' => $countryCode,
                'data_dir' => $result['data_dir'],
                'import_status' => 'success'
            ]
        ], 200);
    }
​
    // Get available countries API
    public function getAvailableCountries()
    {
        $countries = $this->handler->getAvailableCountries();
        return response()->json([
            'code' => 200,
            'message' => 'Success',
            'data' => ['countries' => $countries, 'count' => count($countries)]
        ], 200);
    }
​
    protected function importDataToDatabase(string $dataDir, string $countryCode)
    {
        // Implement data import logic (e.g., JSON to database)
    }
}

Route Configuration

// routes/api.php
use App\Http\Controllers\Api\LocationController;
Route::prefix('location')->group(function () {
    Route::get('available-countries', [LocationController::class, 'getAvailableCountries']);
    Route::post('sync', [LocationController::class, 'syncCountryData']);
});

Unit Testing

Key test cases cover API success/failure, valid/invalid country codes, and file operations. Run tests with:

php artisan test tests/Unit/Location/GithubLocationHandlerTest.php -v

4. Global Deployment & Optimization

Network Adaptability

• Mirror Support: Switch to GitLab/Gitee mirrors via GITHUB_LOCATION_REPO env variable.

• Proxy Configuration: Add proxy settings in config/plugins/location.php for restricted regions.

• Retry Mechanism: Built-in HTTP retries handle global network fluctuations.

Production Best Practices

• Directory Permissions: Set storage/app/location-temp to 0755 with web server ownership.

• Caching: Use Redis for better performance in distributed environments.

• Task Scheduling: Automate syncs with Laravel Scheduler and queues:

  // app/Console/Kernel.php
  protected function schedule(Schedule $schedule)
  {
      $schedule->call(function () {
          $handler = app(GithubLocationHandler::class);
          foreach ($handler->getAvailableCountries() as $code) {
              SyncLocationDataJob::dispatch($code)->onQueue('location_sync');
          }
      })->weeklyOn(0, '02:00')->name('sync-location-data');
  }

• Log Monitoring: Integrate Sentry/ELK for global error tracking.

5. Extension Directions

• Multi-Repo Support: Create an abstract BaseGithubHandler for reusing logic across repositories.

• Version Control: Add data version checks to avoid outdated syncs.

• Frontend Dashboard: Build a Vue/React interface to monitor sync status.