If you use a Select DISTINCT query, you’ll get back a recordset consisting only of the unique records: Red Suppose you had a simple database with these entries: So how does that enable us to eliminate duplicate lines? Well, there’s a kind of database query known as Select DISTINCT what Select DISTINCT allows you to do is select all the distinct (or unique) records in a table. For now, suffice to say that we’re working with the text file C:\Scripts\Test.txt, something we indicate by assigning the appropriate values to the variables strPathToTextFile and strFile: strPathToTextFile = “C:\Scripts\” We won’t spend a lot of time detailing how you treat a text file as if it were a database if you’d like to learn more about that we have a Scripting Clinic column covering that topic in detail. We find this script kind of interesting because we’re using ActiveX Data Objects (ADO) and treating this text file as if it were a database. ObjConnection, adOpenStatic, adLockOptimistic, adCmdText ObjRecordSet.Open “Select DISTINCT * FROM ” & strFile, _ “Extended Properties=””text HDR=NO FMT=Delimited””” “Data Source=” & strPathtoTextFile & “ ” & _ Set objRecordSet = CreateObject(“ADODB.Recordset”) Set objConnection = CreateObject(“ADODB.Connection”) SW, you came to the right place: Const adOpenStatic = 3 You want a script that can weed out all the duplicate lines and provide you with output similar to this: This is one of the lines in the text file. This is yet another line in the text file.
This is one of the lines in the text file.
It’s unlikely, but maybe your file looks like this: This is one of the lines in the text file. To begin with, we assume you have a text file where each line represents a separate record. As to whether this is a better/faster/easier solution than the one we offered before, well, we’ll leave that up to you. And now, thanks to your question, we can take another shot at this. The solution we came up with was simple enough, and it worked just fine we just weren’t convinced that it was the best solution. Awhile back we answered a similar question, one about removing duplicate names from a text file. (Or at least that’s what we tell our manager when he asks why we never actually seem to finish anything: “But boss, never-ending quests take time!”) That’s why we were glad to see your question. You know, to be a Scripting Guy means to embark on a never-ending quest to find the ultimate solution to a given problem.
0 kommentar(er)
