'transaction_id', 'card_acc_id' => 'card_acc_id', 'pan_number' => 'pan_number', 'txn_type' => 'txn_type', 'merchant_id' => 'merchant_id', 'txn_amount' => 'txn_amount', 'booking_date' => 'booking_date', 'trans_ref' => 'trans_ref', 'retrieval_ref_no' => 'retrieval_ref_no', 'stmt_nos' => 'stmt_nos', 'debit_acct_no' => 'debit_acct_no', 'credit_acct_no' => 'credit_acct_no', 'chrg_amount' => 'chrg_amount', 'value_date' => 'value_date', 'stan_no' => 'stan_no', 'trans_status' => 'trans_status', 'proc_code' => 'proc_code' ]; private string $period = ''; private int $processedCount = 0; private int $errorCount = 0; private array $atmTransactionBatch = []; /** * Create a new job instance. */ public function __construct(string $period = '') { $this->period = $period; } /** * Execute the job. */ public function handle() : void { try { $this->initializeJob(); if ($this->period === '') { Log::warning('No period provided for ATM transaction data processing'); return; } $this->processPeriod(); $this->logJobCompletion(); } catch (Exception $e) { Log::error('Error in ProcessAtmTransactionJob: ' . $e->getMessage()); throw $e; } } private function initializeJob() : void { set_time_limit(self::MAX_EXECUTION_TIME); $this->processedCount = 0; $this->errorCount = 0; $this->atmTransactionBatch = []; } private function processPeriod() : void { $disk = Storage::disk(self::DISK_NAME); $filename = "{$this->period}." . self::FILENAME; $filePath = "{$this->period}/$filename"; if (!$this->validateFile($disk, $filePath)) { return; } $tempFilePath = $this->createTemporaryFile($disk, $filePath, $filename); $this->processFile($tempFilePath, $filePath); $this->cleanup($tempFilePath); } private function validateFile($disk, string $filePath) : bool { Log::info("Processing ATM transaction file: $filePath"); if (!$disk->exists($filePath)) { Log::warning("File not found: $filePath"); return false; } return true; } private function createTemporaryFile($disk, string $filePath, string $filename) : string { $tempFilePath = storage_path("app/temp_$filename"); file_put_contents($tempFilePath, $disk->get($filePath)); return $tempFilePath; } private function processFile(string $tempFilePath, string $filePath) : void { $handle = fopen($tempFilePath, "r"); if ($handle === false) { Log::error("Unable to open file: $filePath"); return; } // Get the headers from the first row $headerRow = fgetcsv($handle, 0, self::CSV_DELIMITER); if (!$headerRow) { fclose($handle); return; } $rowCount = 0; $chunkCount = 0; while (($row = fgetcsv($handle, 0, self::CSV_DELIMITER)) !== false) { $rowCount++; $this->processRow($headerRow, $row, $rowCount, $filePath); // Process in chunks to avoid memory issues if (count($this->atmTransactionBatch) >= self::CHUNK_SIZE) { $this->saveBatch(); $chunkCount++; Log::info("Processed chunk $chunkCount ({$this->processedCount} records so far)"); } } // Process any remaining records if (!empty($this->atmTransactionBatch)) { $this->saveBatch(); } fclose($handle); Log::info("Completed processing $filePath. Processed {$this->processedCount} records with {$this->errorCount} errors."); } private function processRow(array $headerRow, array $row, int $rowCount, string $filePath) : void { if (count($headerRow) !== count($row)) { Log::warning("Row $rowCount in $filePath has incorrect column count. Expected: " . count($headerRow) . ", Got: " . count($row)); $this->errorCount++; return; } // Combine the header row with the data row $rawData = array_combine($headerRow, $row); $this->mapAndAddToBatch($rawData, $rowCount, $filePath); } private function mapAndAddToBatch(array $rawData, int $rowCount, string $filePath) : void { // Map the raw data to our model fields $data = []; foreach (self::HEADER_MAP as $csvField => $modelField) { $data[$modelField] = $rawData[$csvField] ?? null; } // Skip header row if it was included in the data if ($data['transaction_id'] === 'id') { return; } $this->addToBatch($data, $rowCount, $filePath); } /** * Add record to batch instead of saving immediately */ private function addToBatch(array $data, int $rowCount, string $filePath) : void { try { // Add timestamp fields $now = now(); $data['created_at'] = $now; $data['updated_at'] = $now; // Add to batch $this->atmTransactionBatch[] = $data; $this->processedCount++; } catch (Exception $e) { $this->errorCount++; Log::error("Error processing ATM Transaction at row $rowCount in $filePath: " . $e->getMessage()); } } /** * Save batched records to the database */ private function saveBatch() : void { try { if (!empty($this->atmTransactionBatch)) { // Process in smaller chunks for better memory management foreach ($this->atmTransactionBatch as $entry) { // Extract all stmt_entry_ids from the current chunk $entryIds = array_column($entry, 'transaction_id'); // Delete existing records with these IDs to avoid conflicts AtmTransaction::whereIn('transaction_id', $entryIds)->delete(); // Insert all records in the chunk at once AtmTransaction::insert($entry); } // Reset entry batch after processing $this->atmTransactionBatch = []; } } catch (Exception $e) { Log::error("Error in saveBatch: " . $e->getMessage()); $this->errorCount += count($this->atmTransactionBatch); // Reset batch even if there's an error to prevent reprocessing the same failed records $this->atmTransactionBatch = []; } } private function cleanup(string $tempFilePath) : void { if (file_exists($tempFilePath)) { unlink($tempFilePath); } } private function logJobCompletion() : void { Log::info("ATM transaction data processing completed. " . "Total processed: {$this->processedCount}, Total errors: {$this->errorCount}"); } }