Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Comparison of two files, except for filenames.

Author
Message
yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 13th Jun 2012 23:17
Hello to the forum members.

I would like to compare two files to see if they are the same except for the filename. For example, an mp3 file named song_1.mp3 and another mp3 named song_2.mp3
Performing this on other file types would be useful as I would like to delete duplicate files which have been renamed.

All the best.
mr Handy
18
Years of Service
User Offline
Joined: 7th Sep 2007
Location: out of TGC
Posted: 14th Jun 2012 11:29
Pro way: Checksum (http://en.wikipedia.org/wiki/Checksum) Calculate for both files and compare them.

Simple way: analyze files.
1. Compare sizes.
2. If sizes are same - get some random bytes, compare them.
3. If bytes are same... you could use pro way here or just take more bytes.

Although there are some programs that do search of nearly same pictures.

«It's the Pony, pony me this, pony me that» — Bronies
«I sell apples and apple accessories» — Applejack
Derpy delivers: watch?v=g4Kgz4Us_RI
yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 14th Jun 2012 14:20
Thanks very much for your reply and the URL mrHandy.

May the force be with you.
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 15th Jun 2012 15:42
Unless you already have checksums of the files you are comparing, you may as well carry out a byte-for-byte comparison of the file data.

If you think about it, you have to read the whole file anyway to generate a checksum for it. At least if you are reading through a file, you can stop checking as soon as you find the first different byte of data.

Green Gandalf
VIP Member
21
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 15th Jun 2012 17:16
Is it possible to use the DOS comp command in some way?
basjak
16
Years of Service
User Offline
Joined: 16th Apr 2010
Location: feel like signing up for mars
Posted: 15th Jun 2012 20:44
You can use hashes at Ian matrix utility . I tried it before and was pretty successful.

Sergey K
22
Years of Service
User Offline
Joined: 4th Jan 2004
Location:
Posted: 15th Jun 2012 21:58
u can compare size and compare file creation date.

Advanced Updater for your games!
yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 16th Jun 2012 03:52
What about loading the files to be compared into two memblocks, if the memblock size is the same for both , a byte by byte bitwise AND
comparison?
yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 16th Jun 2012 06:16
Would this suffice



Rem Project: duplicatecompre
Rem Created: Saturday, June 16, 2012
global count

Rem ***** Main Source File *****
open to read 1, "document1.txt"
make memblock from file 1,1
close file 1

open to read 2, "document2.txt"
make memblock from file 2,2
close file 2

firstfile=GET MEMBLOCK SIZE(1)
secondfile=GET MEMBLOCK SIZE(2)

if firstfile=secondfile
print "the file sizes are the same"
sizeinbytes=(firstfile/8)
print "Size in bytes = "; sizeinbytes
ENDIF

print "the size of file 1 is " ; firstfile
print "the size of file 2 is " ; secondfile

count=0
for x = 0 to sizeinbytes
rem check each byte is the same
if MEMBLOCK BYTE(1, x) = MEMBLOCK BYTE(2, x)
count=count + 0
sync
else
count=count + 1
ENDIF
NEXT x
print count;

rem
rem source to delete file two based on count value
rem


sync
wait key
delete memblock 1
delete memblock 2
yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 16th Jun 2012 10:36
This will work!

Rem Project: duplicatecompre
Rem Created: Saturday, June 16, 2012
global count
sizeinbytes as integer
global firstfile as integer
global secondfile as integer
firsttest as integer
secontest as integer
Rem ***** Main Source File *****
open to read 1, "document1.txt"
make memblock from file 1,1
close file 1

open to read 2, "document2.txt"
make memblock from file 2,2
close file 2

firstfile=GET MEMBLOCK SIZE(1)
secondfile=GET MEMBLOCK SIZE(2)

if firstfile=secondfile
print "the size of file 1 is " ; firstfile
print "the size of file 2 is " ; secondfile
ENDIF


firstfile=firstfile-1
count=0
for x = 0 to firstfile
firsttest = MEMBLOCK BYTE(1, x)
secondtest = MEMBLOCK BYTE(2, x)
if firsttest=secondtest
count=count + 0
else
count=count + 1
ENDIF
NEXT x

print count;

rem
rem source to delete file two based on count value
rem if count 0 then delete filename two
rem


sync
wait key
delete memblock 1
delete memblock 2
Libervurto
20
Years of Service
User Offline
Joined: 30th Jun 2006
Location: On Toast
Posted: 17th Jun 2012 04:26
Since you are just comparing sizes you could read DWORDS instead of BYTES to save time.

Forum Tips:
Surround your code with code tags or highlight your code and click the code button to have them put in for you. This makes your code appear in a collapse-able code box with retained indentation.

You can edit your posts by clicking the link on the bar below you post (to the far left in blue text).

yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 17th Jun 2012 12:36
Thanks for the forum tip. A byte by bye comparison of the memblocks does occur here, so it's just not comparing file sizes.

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 17th Jun 2012 17:16 Edited at: 18th Jun 2012 07:51
yoda333,

He's talking about comparing DWORD (a pack of 4 bytes) rather than just individual BYTES. Which may potentially make the process up to 4 times quicker, with minimal effort.

Another idea would be to split the loop, into a control and inner loops. The inner could compare say 32/64 bytes(or longer ) blocks and outer loop steps through the hunks, aborting when chunk doesn't match. It'd need a second loop to handle any left over bytes when the size isn't a multiple of the chunk size.

This helps reduce the loop/function call and comparison overhead as the files get larger. Dunno how well that'd translate to DBPRO though.

Pseudo Code



Not sure if the command is MEMBLOCK DWORD or MEMBLOCK INT in Dbpro. So this is theory only.

GIDustin
18
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Jun 2012 20:00
I use IanM's hash function to return an MD5 of the file.

yoda333
14
Years of Service
User Offline
Joined: 3rd Apr 2012
Location:
Posted: 18th Jun 2012 02:50
Thanks very much to everybody for their suggestions on this one, it has given me plenty to work with. You are all truly great.

May the force and lotto gods be with you.
MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 18th Jun 2012 11:38
I really need to get into my programming I feel my brain is being projected to the moon... I have been thinking about making this for some time now..

hope to make a specific one myself too

IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 19th Jun 2012 15:50
Don't use a hash to compare files for equality.

If the hashes for your files differ, then you know the files are different. If the hashes match, then you know only that they MAY be the same.

If you have multiple files, then you can use the file hashes to eliminate different files quickly, then use Kevins example to compare actual file data where the hashes match.

You may find that mapping the file into a bank rather than reading it into a memblock may be faster (it cuts down on the amount of copying in memory - YMMV).

Login to post a reply

Server time is: 2026-07-08 03:49:01
Your offset time is: 2026-07-08 03:49:01