Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Geek Culture / Bulk Find and Replace/Delete Program Like Ultraedit

Author
Message
Epimetheus
20
Years of Service
User Offline
Joined: 5th Oct 2003
Location: Naples, Florida
Posted: 23rd Feb 2014 14:42
Link
I don't know if something like Ultraedit or Ultracompare can do this but I need to be able to take a list or entries, and then search another list by those entries to find lines containing entries from the first list. For example, I would take a list of domain names and then check to see which lines contain those domain names(or anything else, cities, zip codes, etc.) on another list of urls. This would work similar to a bulk find feature. After finding the entries I need to extract and/or delete the matching lines.

Is this possible with Ultraedit, Ultracompare, or anything else?
Back to top
Profile PM Email
Jeku
Moderator
20
Years of Service
User Offline
Joined: 4th Jul 2003
Location: Vancouver, British Columbia, Canada
Posted: 23rd Feb 2014 17:27
Link
Not in Ultra-edit as far as I know. I think it wouldn't be difficult to make your own utility for this, though.


Senior Developer - CBS Interactive Music Group
Back to top
Profile PM Website
Epimetheus
20
Years of Service
User Offline
Joined: 5th Oct 2003
Location: Naples, Florida
Posted: 23rd Feb 2014 18:06
Link
@Jeku

the main problem would be speed when working with large databases, that would require multi-threading which makes this somewhat more complex
Back to top
Profile PM Email
Jeku
Moderator
20
Years of Service
User Offline
Joined: 4th Jul 2003
Location: Vancouver, British Columbia, Canada
Posted: 24th Feb 2014 07:27
Link
Well Ultraedit doesn't work with databases anyways, as far as I know By the sound of your original question it seemed like you'd be analyzing text files, so it wouldn't be hard to roll your own.


Senior Developer - CBS Interactive Music Group
Back to top
Profile PM Website
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 24th Feb 2014 10:23
Link
In Notepad++ you can do a batch search and replace on files.
If you use the Regular Expression mode, you can essentially program your search.

The slight issue is whether you have used regular expressions. If you haven't there's a bit of a learning curve But there's lots of tutorials out there.

ten-net consulting limited
Back to top
Profile PM Email
nonZero
12
Years of Service
User Offline
Joined: 10th Jul 2011
Location: Dark Empire HQ, Otherworld, Silent Hill
Posted: 24th Feb 2014 11:17
Link
I'll also agree on the DIY option. The time and money to procure special tools wouldn't be worth a day's grind learning to make your own. Don't worry about multithreading, you'll be surprised how fast a single thread can handle a db. Remember modern CPUs short-circuit mutiple conditions once the outcome is obvious, ie with OR that's the first TRUE and with AND that's the first FALSE.

ver 7.5 /// int 145 /// str 45 /// dex 85 /// end 200 /// mat 3
Back to top
Profile PM Email Website
BiggAdd
Retired Moderator
19
Years of Service
User Offline
Joined: 6th Aug 2004
Location: != null
Posted: 24th Feb 2014 20:32
Link
I'm not really sure what exactly you are looking for, but I can agree with what people are saying about writing a script to do this.

If the files are massive, you could always import them into a relational database if you need to do lots of data analysis.

Back to top
Profile PM
Epimetheus
20
Years of Service
User Offline
Joined: 5th Oct 2003
Location: Naples, Florida
Posted: 25th Feb 2014 20:04
Link
@BatVink

I'm familiar with regex and notepad++, but for large lists this would very very slow or cause a crash, for example, if you had a regex like item 1 | item 2 | item 3 | ... | item 1000 it wouldn't be the most feasible solution
Back to top
Profile PM Email
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 25th Feb 2014 21:08
Link
If it's text files, then Regex is still your best answer. Regex is "compiled" into a fast scan that beats a straight Find & Replace by a huge magnitude.

So your best compromise might be a VBA script in Excel. List your requirements in a worksheet, traverse the sheet and convert each one to a Regex expression (built into VBA).

I can say from experience this works well. I have done this (in VB.NET but it's similar) to extract meta data from Java programs. I had around 20 Regex statements, 8,000+ files and 16 million lines of code, it completed the task (including writing to the database) in about 24 minutes.

ten-net consulting limited
Back to top
Profile PM Email
Jeku
Moderator
20
Years of Service
User Offline
Joined: 4th Jul 2003
Location: Vancouver, British Columbia, Canada
Posted: 26th Feb 2014 00:52
Link
I regularly write scripts at work to go through millions of database records, and the time estimates can vary from 5 minutes to a few hours. If you want to feed your data into a DB and then write some PHP to whip through them, I too would recommend regex for pattern matching. It's really the best way.

I recommend http://regex101.com/ for building your regex patterns... this site has saved my butt more than once!


Senior Developer - CBS Interactive Music Group
Back to top
Profile PM Website
Back to top

Login to post a reply

Server time is: 2024-05-04 18:45:17
Your offset time is: 2024-05-04 18:45:17