Programmer's File Editor With Change Tracking?

passionfingers writes "My business users regularly have to tweak large (>32MB text) data files manually. Overlords charged with verifying the aforementioned changes have requested that the little people be provided with a new file editor that will track changes made to a file (as a word processor does). I have scouted around online for such an animal, but to no avail — even commercial offerings like UltraEdit32 don't offer such a feature. Likewise on the OSS side of the fence, where I expected a Notepad++ plugin or the like, it appears that the requirements to a) open a file containing a large volume of text data and b) track changes to the data, are mutually exclusive. Does anyone in the Slashdot community already have such a beast in their menagerie? Perhaps there is there a commercial offering I've missed, or could someone possibly point me to their favorite (stable) OSS project that might measure up?"

vi/emacs/eclipse/whatever + svn?

[FooAtWFU (699187)]

In the open-source world you edit your text file, finish for a bit, save it, and check it back into your favorite form of source control. You can then look at the diffs between various revisions.

Emacs - ~/.saves directory

[JBrow (668684)]

Aside from using version control, I also have Emacs set up with an extensive save history. My settings are:

; backup
(setq make-backup-files t)
(setq
backup-by-copying t
backup-directory-alist '(("." . "~/.saves"))
delete-old-versions t
kept-new-versions 6
kept-old-versions 2
version-control t)

rent a geek

[gabec (538140)]

The OP has a lot of things working against him:

1) Obviously never heard of version control like (CVS, SVN, etc.). This is excusable, as setting up and keeping with your very own SVN repo isn't the top of people's lists, and it's entirely possible to not have heard of such a beast.

2) Accepting the commandment of "thou shalt manually wade through gobs of data in a text file means he's no programmer. This is also perfectly fine, but means he's not the right person to be changing this file anyway.

3) Is under the impression that revision history could conceivably be hidden within said plain text file, implying the OP doesn't understand basic file formats, which confirms that he is not merely the wrong person but absolutely the last person you want manually mucking with your data files.

Therefore...

My suggestion: Forward the request to your IT department (or rent a geek).

Re:

[Hotawa Hawk-eye (976755)]

3) Is under the impression that revision history could conceivably be hidden within said plain text file, implying the OP doesn't understand basic file formats, which confirms that he is not merely the wrong person but absolutely the last person you want manually mucking with your data files.

Revision history can't be hidden within a plain text file?

\documentclass{article}
\begin{document}
This is a \LaTeX document. It can have comments in it.
% Here is a comment that will not be present in the LaTeX document

Re:

[Chuckstar (799005)]

But that only works if the file format supports comments.

Re:

[AigariusDebian (721386)]

Storing version information in a complimentary file is the very basic description of what version control systems do.

Give your users an editor with WebDav support and make them edit the files directly in the SVN repository using a HTTP WebDav repository feature of SVN.

Version control

[Just Some Guy (3352)]

You don't solve that by changing editors. You solve that by standardizing on a version control system and using it. For example, you could stick a Subversion server somehwere on your network, then install TortoiseSVN on your client machines. When you edit the file, you commit it with a log message of why you made the changes.

This is infinitely preferable to forcing all of your developers to use one specific editor (which is the universal holy war among programmers), when all of them will be half-assed at best when compared to real version control.

Re:Version control

[Shaitan Apistos (1104613)]

Some editors, netbeans for example, even have visual cues in the margin indicating what you've changed from the svn revision you've checked out since your last commit.

Re:

[Shados (741919)]

Yeah, I just don't think a 32 meg file in netbeans with svn and the change tracking in the margin will be a very efficient way of editing the file though, considering these features in most such IDEs tend to start choking after a few hundred kilobytes.

Re:Version control

[markov_chain (202465)]

You must switch to version control, urgently! [thedailywtf.com]

Re:Real version control

[pjt33 (739471)]

It's not quite as straightforward as that [kerneltrap.org], but there's a valid objection there, and I take the point.

Of course, the real problem is the idea that users manually editing a 32MB file on a regular basis is at all a sensible idea.

Re:Real version control

[Kent Recal (714863)]

It's not "unusual", it's simply nonsense.
If your business process requires non-techie users to edit and share 32M-sized textfiles regularly then your business process is broken. The whole approach is broken beyond repair, the obvious solution would be to build some kind of frontend for the users.

Re:

[pjt33 (739471)]

When I was young, we used RCS and we liked it! As the state of the art changes, so do the requirements to stay at the top. It's possible that SVN 1.5 qualifies as real version control by modern standards - I'll find out when it reaches my somewhat conservative distro - but previous versions have poor support for merging.

I'm also rather unhappy at SVN this week because it managed to get itself in a horribly confused state in which it told me I needed to run svn cleanup to fix some locks, but running svn clea

source control?

[MariusBoo (883340)]

Are the overlords retarded? Just use source control (svn?). "Track changes" is evil.

diff -Nrau

[red_dragon (1761)]

What's so bad about diff that you're not using it already? Certainly it would be a better choice for dealing with multiple text files: make a working copy (which can be an entire directory tree), do your changes on the working copy, and then run diff against the original and working copies. And I bet that there's an Emacs mode for that too.

CVS/SVN?

[Jonah Hex (651948)]

I understand you want an all-in-one, however I believe that most "programming" editors can hook into some sort of change management program. I use one locally with my AutoIT3 scripts and the SCiTE editor, every time I compile it asks me for a "changes/reason" and enters that into my own local source management.

SCiTE [scintilla.org]

SCiTE for AutoIT [autoitscript.com] with screenshots

CVS/SVN wrapper for SCiTE [autoitscript.com] with screenshots and instructions

Jonah HEX

Custom tool

[CaseyB (1105)]

In addition to joining the chorus that will suggest you use version control, I'll put in a suggestion to write a custom tool to view and make specific changes to the file. Multiple users editing *data* files by hand, with no validation, is silly.

change the process

[DragonTHC (208439)]

if your business users have to edit 32MB text files by hand, someone is a moron.

Someone didn't know what they were doing in the first place.

Change the process to make sense.

Perhaps those "files" should actually be in a database. SQL makes automatic manipulation quite easy.

Re:change the process

[blincoln (592401)]

Perhaps those "files" should actually be in a database. SQL makes automatic manipulation quite easy.

Exactly. That also makes it easy to build a change audit logging table.

Re:change the process

[DragonTHC (208439)]

come to think of it, it sounds like you're asking us how to best manage spam hit lists.

Re:

[cstdenis (1118589)]

Thats silly. Spammers don't edit their lists. They just blast it out to all, valid or fake.

Re:

[qoncept (599709)]

That's a pretty short-sighted comment. It's very easy to end up doing something that outwardly looks totally foolish. Maybe these files started out about 4kb (and as I imagine this example, I'm remembering when it happened in my project) and unforseen changes caused it to gradually grow to 32mb. Hand editing from the beginning wasn't a big deal, but now it is.

Which brings up the issue of changing this "process." Have you worked in a fast paced environment with limited resoures? Processes don't just chang

Re:

[Baddas (243852)]

Sounds like they're in a non-sustainable environment. Once they run off the rails, or someone gets sick, or someone puts a comma where they shouldn't, the whole house of cards comes crashing down.

Risk management via creating space to improve the process is surely a better option than that, right?

Source Control

[Imagix (695350)]

I'll echo the same sentiments as everybody else... use source control. In addition, you're asking for the impossible. Word Processors can do "change tracking" within a file because the file contains more stuff than simply the text that you type in. Based on your description, the file you're talking about is simple text data, and has no provisions for storing "previous versions" directly within the text file. Where were you expecting this additional information to reside? And if it's in the same file, you would also have to change whatever tools you're using to process that file to be aware of all of this new (and to that tool, useless) information within the file.

diff

[by Anonymous Coward]

I just wrote one for you, and it is even using your favorite editor:
cp $1 $1.bak
`$EDITOR $1`
diff $1.bak $1

Re:RCS

[SiliconSlick (458714)]

And here's one that will track previous changes as well as well as preventing edits by two people at once...

$ cat edit.sh
#!/bin/sh
co -l $1
$EDITOR $1
ci $1
co $1

Wiki?

[mi (197448)]

Sounds like Wiki may be the best... It is easy enough to split the document into sections, which can be edited concurrently. It keeps the history available. And the format is (almost) text.

Pick MediaWiki [mediawiki.org] (the same software, that powers WikiPedia) or any other implementation (some may be easier to operate on a small LAN, and/or be able to export pure text, etc.)

Simple, switch to VMS!

[antifoidulus (807088)]

Automatic file versioning built right into the file system! Problem solved! Next!

Re:Simple, switch to VMS!

[UnderCoverPenguin (1001627)]

If the users are using Linux, there is also ext3cow, which was discussed on /. (http://linux.slashdot.org/article.pl?sid=07/05/02/0413253&from=rss), the newly announced Tux3 (http://lkml.org/lkml/2008/7/23/257), Wayback (http://wayback.sourceforge.net/) and others.

Um, Eclipse?

[Progman3K (515744)]

I seem to recall Eclipse saves your edit history

Does the file format support it?

[Richard_at_work (517087)]

a new file editor that will track changes made to a file (as a word processor does)

A large part of the 'as a word processor does' feature is that the file format itself supports it - all the changes are saved right along side each other and the word processor picks and chooses what to display to you at any particular point in time. If your file format does not support change tracking, you need to look at external change tracking solutions such as SVN/CVS or something on the file system level.

Fix The Bug In The Overlords Heads

[Bob9113 (14996)]

Overlords charged with verifying the aforementioned changes have requested that the little people be provided with a new file editor that will track changes made to a file (as a word processor does).

Apparently your Overlords' heads have some fault code and need to be debugged.

Revision tracking is accomplished with revision tracking software.

You may think it is easier to just do what they are telling you to do, but in the long run it is not. They need to understand the difference between editing and revision tracking. If they do not, they will not be getting what they are looking for.

There are a variety of methods for tracking revisions, from diff files to Subversion. Word processors store delta histories in the document, which is a poor place to store such things because it is insecure and liable to corruption.

Then again, if you're doing hand edits to 32M text files, you probably are working in a company with less comprehension of information science than the City of San Francisco [slashdot.org] (alas, my beloved home city, but boy do they have their heads up their nether regions).

Seriously - you may not like this answer, but it is the only correct one. Fix their understanding of revision tracking.

diff

[slashflood (697891)]

Keep a copy of the original file and make a diff [gnu.org]. Am I missing something?

Re:AskSlashdot: "Please Do My Work For Me"

[Yurka (468420)]

Many of the Slashdotters actually pay for the opportunity to, among other things, answer questions like that. If you do not derive satisfaction from doing it, just step aside, no one is forcing you to post.

That's not fair

[Scotteh (885130)]

When you don't know how to do something, where do you go? I usually got to Google first and look around. Sometimes I find something, sometimes I don't. When I can't find anything via Google, where then do you turn? A co-worker? A friend? Maybe an online community of people who would know? The latter sounds like the most promising to me.

A company isn't concerned with how you find the answer (provided you come by it legally), they just want an answer as soon as possible. The faster, the cheaper the solution man-hours-wise.

I wouldn't be surprised if you've posted for help on an online forum or asked a friend for help with something. It's no different. It's all part of research.

Ask Google, then ask Slashdot

[tepples (727027)]

When I can't find anything via Google, where then do you turn?

You could ask Slashdot or some other forum, but provide an example of a Google query that you thought was promising but didn't turn up anything relevant. This way, others can see that you at least tried, and people can use why the queries didn't work as a starting point for conversation.

Re:That's not fair

[spazdor (902907)]

their domain name used to be "expertsexchange.com".
Then one day that address suddenly redirected to "experts-exchange.com" You could almost feel the webmasters smacking themselves in the forehead.

Re:That's not fair

[wilder_card (774631)]

I've got a list of "worst URLS" around somewhere that's got expertsexchange.com on it. Other memorable entries included an artist's site, speedofart.com, and the Mole Station Nursery, a wildlife non-profit, at molestationnursery.com.

Re:That's not fair

[spazdor (902907)]

http://burntherapist.com/ [burntherapist.com]

Re:That's not fair

[A440Hz (1054614)]

Don't forget-- the company Pen Island decided to use their name (without dashes) as their URL. Capitalization being rare in URLs, it didn't work so well.

Re:That's not fair

[hotdiggitydawg (881316)]

Neither did the URL for the Italian branch of a multinational energy company: "PowerGen Italia"...

Re:

[FiloEleven (602040)]

I've never had to turn off cookies or anything to view answers on Experts Exchange. Just scroll down past the masked answers and past all the categories: lo and behold, all of the answers are there in plain sight! This is the case for Firefox, at least.

Re:

[Fez (468752)]

They show up fine for me, just keep scrolling down. When I land on one of their pages from a google search, I see all of their gibberish posts but underneath all that, way down the page, I see the full text answers.

Re:

[$1uck (710826)]

Bad Idea... I don't think google or any search engine should index things that aren't public. I think its retarded the way the search engine will point you to something you can't see or use w/o signing up. Really I wish google didn't index expert sex change. Its just more noise in the results.

Re:That's not fair

[Oroki (1197885)]

Just scroll to the bottom of the page; all the answers are there.

Re:

[somersault (912633)]

He points out that he has had a look himself, but he doesn't seem to have heard of version tracking software, or cpmsoders it overly complex for their needs. Personally I've never used any myself, but it sounds about right for this type of task.

Re:AskSlashdot: "Please Do My Work For Me"

[by Anonymous Coward]

> cpmsoders

I think you need to move your keyboard about this much (___) to the left. The scary thing is it made perfect sense!

Re:

[somersault (912633)]

Oops! That's quite imcjaracjterostoc of me... my apologies.

Re:

[jimwelch (309748)]

"not to sound like a..." TOO LATE!

Why complain? Too many people on slashdot are whiners! If you don't want to help, don't respond! That is the beauty of "open source". Help if you want or don't.

I have been programming for over 30 years and am considered a guru by my peers. But I know my limitations. For example I would not touch a SAP project with out help, because I have no experience.

Maybe the solution for everyone, is to have a category for "I need help" that people like you can tell slashdot not to show

Re:

[ianare (1132971)]

well considering many of us here are at work, we are getting paid for it.