daengdeni/webstatement
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(closing-balance): implementasi unique_hash dan insertOrIgnore untuk eliminasi duplikasi

Browse Source 
Perbaikan masalah duplikasi pada laporan penutupan saldo dengan pendekatan hash unik dan query insert yang toleran terhadap duplikasi.

Perubahan:
- Tambah kolom `unique_hash` pada tabel `processed_closing_balances` (via migrasi `2025_07_31_035159_add_unique_hash_field_to_processed_closing_balances_table.php`)
- Tambah field `unique_hash` ke `$fillable` pada model `ProcessedClosingBalance`
- Update logika generate unique key di `prepareProcessedClosingBalanceData()` menggunakan `md5(trans_reference + '_' + amount_lcy)`
- Query pencarian duplikasi berdasarkan `unique_hash`, bukan `trans_reference` saja
- Ganti `insert()` dengan `insertOrIgnore()` untuk mencegah error saat insert duplikat data

Dampak:
- Duplikasi data dihindari secara efektif lewat hash unik
- Tidak ada error meski data duplicate ditemukan, karena query otomatis mengabaikannya
- `trans_reference` yang sama tetap valid selama nilai `amount_lcy` berbeda
- Data laporan lebih konsisten dan terhindar dari konflik constraint
This commit is contained in:
Daeng Deni Mardaeni
2025-07-31 11:06:11 +07:00
parent 13e077073b
commit 710cbb5232
3 changed files with 37 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
app/Jobs/GenerateClosingBalanceReportJob.php
View File

@@ -177,7 +177,8 @@ class GenerateClosingBalanceReportJob implements ShouldQueue
$insertChunks = array_chunk($allProcessedData, 1000);
foreach ($insertChunks as $chunk) {
DB::table('processed_closing_balances')->insert($chunk);
// Menggunakan insertOrIgnore untuk mengabaikan duplikasi dan mencegah error unique constraint
DB::table('processed_closing_balances')->insertOrIgnore($chunk);
}
Log::info('All processed data inserted successfully', [
@@ -527,35 +528,17 @@ class GenerateClosingBalanceReportJob implements ShouldQueue
private function prepareProcessedClosingBalanceData($transactions, &$runningBalance, &$sequenceNo): array
{
$processedData = [];
$seenReferences = [];
// Buat lookup array untuk validasi cepat - periksa kombinasi trans_reference + amount_lcy
$existingReferences = ProcessedClosingBalance::where('account_number', $this->accountNumber)
->where('period', $this->period)
->where('group_name', $this->groupName)
->get(['trans_reference', 'amount_lcy'])
->map(function ($item) {
return md5($item->trans_reference . '_' . $item->amount_lcy);
})
->toArray();
// Buat lookup array untuk validasi cepat
$existingReferences = array_flip($existingReferences);
foreach ($transactions as $transaction) {
// Validasi duplikasi berbasis trans_reference + amount_lcy + booking_date
$uniqueKey = md5($transaction->trans_reference . '_' . $transaction->amount_lcy);
if (isset($seenReferences[$uniqueKey])) {
Log::warning('Duplicate transaction skipped', [
'trans_reference' => $transaction->trans_reference,
'amount_lcy' => $transaction->amount_lcy
]);
continue;
}
$existingReferences = ProcessedClosingBalance::select('unique_hash')
->where('unique_hash', $uniqueKey)
->get();
// Periksa kombinasi trans_reference + amount_lcy, bukan hanya trans_reference
if (isset($existingReferences[$uniqueKey])) {
if ($existingReferences->count() > 0) {
Log::warning('Transaction already exists in database', [
'trans_reference' => $transaction->trans_reference,
'amount_lcy' => $transaction->amount_lcy
@@ -610,7 +593,7 @@ class GenerateClosingBalanceReportJob implements ShouldQueue
'created_at' => now(),
'updated_at' => now(),
// Tambahkan hash unik untuk memastikan keunikan
'unique_hash' => md5($this->accountNumber . $this->period . $this->groupName . $transaction->trans_reference . $transaction->amount_lcy . $transaction->booking_date)
'unique_hash' => md5($transaction->trans_reference . '_' . $transaction->amount_lcy)
];
}