[UTIL][24MAY10]PreKitchen Tool (Sort, Clean, Merge and create folders)

[TehPenguin]

(Formerly "Dump Sorter" and "XipDumpSort")

New and Improved - Much more functionality than the old one!
I'm trying to make this a general cleaning\sorting\worker app for processing dumped folders. If you'd like a feature added, please let me know.

What is it?
This is a simple program designed to take the hard work out of rearranging, cleaning and merging dumped ROMs with Kitchens.

What can it do?

Clean - Removed all files with the same name as their directories (This is very useful when converting files to modules)
Sort - Files and Folders can be sorted by:

Ervius Kitchen's folder format
osKitchen's folder format
By comparing to a donor folder (should work with any kitchen)

Unsort from:

Ervius Kitchen's folder format
osKitchen's folder format

Merge missing file and folders from a donor directory (useful when the dump is missing files\folders) (SVN Directories are ignored ('.svn' and '_svn'))


How to install
Click the 'Install' button at http://vidmes.com/dumpsorter/publish.htm
(In order to use the 'launch' link, you need to be using IE)
(Alternatively, just use the zip below)

Notes
Merging is done last, so if the donor directory is 'dirty', this 'dirt' will be copied across

This requires .NET 3.5

Also attached is the source code (for anybody interested) - the project was built using Visual Studio 2010, so it may not open in older versions of VS

Known Issues



Things to do

Get a logo\icon
Ability to pull data from archives (eg zip, rar and 7 zip)


Changelog
Version numbers: <installer version> (<executable version>)

* 0.3.0.3 (0.3.3796.19478)
Fixed: Settings in ComboBoxes would not save (e.g. language, resolution, etc)
WARNING: Upgrading to 0.3.0.3 will wipe any of your current settings

* 0.3.0.2 (0.3.3789.38088)
Fixed: Bug where the 'Browse' button for selecting the donor folder to clone didn't do anything

* 0.3.0.1 (0.3.3734.42067)
Added: Support for OndraStar's osKitchen (sorting, unsorting, creating blank folders)

* 0.3.0.0 (0.3.3733.25038)
Added: Ability to unsort from Ervius format
Added: Ability to only sort DPI\RES\LANG that you actually want

* 0.2.0.1 (0.2.3518.38275)
Added: Ability to invert log
Added: Update log on the fly (instead of at the end)
Fixed: Progress bar go to 100% when complete (or reset afterwards)

* 0.2.0.0 (0.2.3516.36999)
Added: Ability to create a default folder layout for Ervius SYS

* 0.2.3516.24314
Added: About box
Fixed: Merger now ignores SVN directories ('.svn' and '_svn')

* 0.2.3516.23053
Added: Cleaner (remove files with same name as directory)
Added: Sorter for Ervius SYS format
Added: Merger (file and folder)
Fixed: Sorter now removes 'unsorted' directory if it is empty

* 1.0.3509.31117
Fixed: Browse buttons now re-enable after sorting is complete

* 1.0.3497.39310
Initial release

[kancrutt]

thanx Mr Moderator
its very usefull :)

[jmckeejr]

WOW!! This is an awesome tool. I'm definitely gonna be giving it a try. Thanks

EDIT: Works perfect ;)

[ivanmmj]

This thing is awesome. It's perfect for organizing SYS folders for any kitchen based on ervius's visual kitchen.

[DarkAngelFR]

Hello,
Thanks for this tool, it's really nice!!!
Just small thing added to your code :
void bgwDoSorting_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
txtLog.Text = (string)e.UserState;
pgbrStatus.Value = (e.ProgressPercentage > 100) ? 100 : e.ProgressPercentage;
}
and :

//old : bgwDoSorting.ReportProgress(workDone * 100 / workTotal);
bgwDoSorting.ReportProgress(workDone * 100 / workTotal, result);

I also reverse the log to have latest lines on top (log+newline instead of newline+log)
:p

Thanks for the great job!!

[SOHKis]

very nice, thanks for sharing!

[TehPenguin]

Hello,
Thanks for this tool, it's really nice!!!
Just small thing added to your code :
void bgwDoSorting_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
txtLog.Text = (string)e.UserState;
pgbrStatus.Value = (e.ProgressPercentage > 100) ? 100 : e.ProgressPercentage;
}
and :

//old : bgwDoSorting.ReportProgress(workDone * 100 / workTotal);
bgwDoSorting.ReportProgress(workDone * 100 / workTotal, result);

I also reverse the log to have latest lines on top (log+newline instead of newline+log)
:p

Thanks for the great job!!

I was considering that, but I think you'll find it will run a bit slower (due to having to constantly update the log box). Then again, if you have a dual processor machine (and who doesn't?), you probably won't notice a difference.

[jmckeejr]

Any chance someone would also be interested in writing a batch file or simple program to delete duplicate files inside of a folder? ie: if there happened to be files inside of the module folder as well as the actual module files. So for example "tapres.dll" module folder for some reason has "tapres.dll" file inside. Would need to got hrough whole SYS though.

[BesFen]

I am testing it now....

[ggpark]

:):):):)Pretty cool set of resources...thanks
Point of Sale (http://www.mpos.net/). (http://greenlife.shoutpost.com/)

[TehPenguin]

Any chance someone would also be interested in writing a batch file or simple program to delete duplicate files inside of a folder? ie: if there happened to be files inside of the module folder as well as the actual module files. So for example "tapres.dll" module folder for some reason has "tapres.dll" file inside. Would need to got hrough whole SYS though.

Sure

Maybe I can change my sorter into a cleaner as well...

Any other 'common tasks' I should include?
Perhaps a tool to merge missing folders into the sorted folder? (eg if the new SYS or OEM is missing stuff from the old one)

[DarkAngelFR]

...Any other 'common tasks' I should include?...
Good ideas...
Maybe I add one : default template folder, for example specify DPI, RESV/H and Lang code(s) then go ;) should be usefull when adding a new language to an existing kitchen who don't have the full folder structure yet for the new language :rolleyes:
If you need help do not hesitate!

PS: about performance, as you said everyone have dualcore computers at least so backgroundworker + log refresh is not slowing the process, and if it is it's not really visible (I'm not working on 1000000 files for 1000 languages so...)

[TehPenguin]

I've rebuilt the backend to be more 'flexible' (ie custom workers and sorters).

I will build the cleaner and the extra sorter some time soon and have it posted up - don't expect anything too soon though...

Attached is the updated source code
(You should be able to tweak the .sln file to work in Visual Studio 2008)

[sIiiS]

Very good application but I can not use it, can you set it to work with .NET 2 ?

[TehPenguin]

Very good application but I can not use it, can you set it to work with .NET 2 ?

I suppose....

Although why not install .NET 3.5?

[sIiiS]

I suppose....

Although why not install .NET 3.5?
Because .Net3.5 has big size for me and members like me to download and install and if I install it, can I remove my .NET2 and .NET1 or need them too for other programs?
Now I have .NET1,2,3 but not .NET3.5 :(

[DarkAngelFR]

.Net 3.5 include 2.0 but you don't need to remove 2.0 before installing 3.5...
If you use windows update it will be big but not as big as if you download the full package from microsoft (it will just take kind of missing parts)
I like the way microsoft made all this framework, you can setup 3.5 without any problem to your existing 2.0 (or 1.1) applications.....more and more programs are using it so it's not a time loosing install for me..

[TehPenguin]

New version released!

Dump Sorter now has all of the features requested.

From the changelog:
* 0.2.3516.24314
Added: About box
Fixed: Merger now ignores SVN directories ('.svn' and '_svn')

* 0.2.3516.23053
Added: Cleaner (remove files with same name as directory)
Added: Sorter for Ervius SYS format
Added: Merger (file and folder)
Fixed: Sorter now removes 'unsorted' directory if it is empty

[+ Que PPC]

Nice i saw your announcement in the release and came to see it... downloading and testing in the next release i seton the kitchen... maybe the tutorials will be useless :p... Thanks a lot man!!!

[DarkAngelFR]

hey there :)
Thanks for this update, will test today...
Missing screenshots on 1st post ;) lol
what about using ClickOnce so we can have auto updates of the program? If you need help hosting I may help.... :rolleyes: I love programs I like to be auto updated and it's really a small thing to do with VS (never tested with 2010)
Keep the good work, looks like we are coming to a complete "pre-kitchen tool"....

[TehPenguin]

hey there :)
Thanks for this update, will test today...
Missing screenshots on 1st post ;) lol
what about using ClickOnce so we can have auto updates of the program? If you need help hosting I may help.... :rolleyes: I love programs I like to be auto updated and it's really a small thing to do with VS (never tested with 2010)
Keep the good work, looks like we are coming to a complete "pre-kitchen tool"....

As much as ClickOnce is awesome, I'd need a server with a valid SSL Cert and .NET 3.5 installed on it - neither of which is going to happen on my budget :(

I'm currently working on your suggestion to build a blank SYS folder structure - once I've done that, I'll update and put up screenshots

[DarkAngelFR]

As much as ClickOnce is awesome, I'd need a server with a valid SSL Cert and .NET 3.5 installed on it - neither of which is going to happen on my budget :(
hmm... I've been using clickonce on an Apache server, without any .net framework on it (linux based server). About SSL Cert it's not really mandatory (it's cleaner but optional). If you want I can do a clickonce publication of the current release just to show...
I'm currently working on your suggestion to build a blank SYS folder structure - once I've done that, I'll update and put up screenshots
Thanks!

[TehPenguin]

hmm... I've been using clickonce on an Apache server, without any .net framework on it (linux based server). About SSL Cert it's not really mandatory (it's cleaner but optional). If you want I can do a clickonce publication of the current release just to show...

Ha! You're right - my mistake...

Now published via ClickOnce: http://vidmes.com/dumpsorter/publish.htm
(I've also added the functionality to create blank folder structures)

[DarkAngelFR]

Hello again!
Just quick tested (sort & blank folder generator)...
Works like a charm!!!
Really nice to update kitchen or to "generate" new one!!!
I found a very small thing : when job done the progress bar is not at 100% and still "blink"...maybe adding a reset of the progress bar in the work complete ;)
Also small thing (ok it's stupid) : click on Go again will try to rearrange folders already sorted, this gives funny things and it becomes hard to find back its files... I used this like that : sort a folder, add 2/3 new folders inside (missing folders for example), resort : oups..... (okey this is not a "normal way to use the tool")
Will test the merge & clean options later (i'm @work...)
see ya' and thanks for this really usefull tool!! (clickonce rocks)

[DomSim]

ummm install link goes to this page,
http://vidmes.com/dumpsorter/XIPDumpSort.application
with nothing to download :confused:

[TehPenguin]

I found a very small thing : when job done the progress bar is not at 100% and still "blink"...maybe adding a reset of the progress bar in the work complete ;)

Good point - I'll add it for the next version

Also small thing (ok it's stupid) : click on Go again will try to rearrange folders already sorted, this gives funny things and it becomes hard to find back its files... I used this like that : sort a folder, add 2/3 new folders inside (missing folders for example), resort : oups..... (okey this is not a "normal way to use the tool")

This is quite hard to prevent. Perhaps I should add a check to warn the user if there is less than 5 sub-folders in the target folder - its not the most elegant solution, but its probably the easiest

Will test the merge & clean options later (i'm @work...)
see ya' and thanks for this really usefull tool!! (clickonce rocks)
I look forward to your report!
(And thanks again for the ClickOnce suggestion)

[TehPenguin]

ummm install link goes to this page,
http://vidmes.com/dumpsorter/XIPDumpSort.application
with nothing to download :confused:

You need to either:
* Use IE
* Click the 'Install' button (not the 'launch' link)

Sorry, this is a limitation of Microsoft's ClickOnce technology (which, apart from the issue you had, is freaking awesome)

[DomSim]

You need to either:
* Use IE
* Click the 'Install' button (not the 'launch' link)

Sorry, this is a limitation of Microsoft's ClickOnce technology (which, apart from the issue you had, is freaking awesome)

alright used ie (blah!) and worked, thank you very much!

[ivanmmj]

This thing is just getting better and better. ^_^

[DarkAngelFR]

Hello,
I tested the merge function and looks to do what it's supposed to do ;)
Didn't test the clean method but some guys did this (have a look here (http://forum.xda-developers.com/showthread.php?p=4358129) )
For me you now need to add 3 things : option to have log in the opposite way (newer events on top), fix the "job done" progress bar issue, lock the "start" button while job is not done (maybe it's already ok, running too fast to be tested here)...Then you need a beautifull icon for the program lol (I'm not a designer guy so won't help on this ;))

I love your program and use it a lot to prepare kitchens fast! THANKS!

[TehPenguin]

For me you now need to add 3 things : option to have log in the opposite way (newer events on top), fix the "job done" progress bar issue, lock the "start" button while job is not done (maybe it's already ok, running too fast to be tested here)...Then you need a beautifull icon for the program lol (I'm not a designer guy so won't help on this ;))

I've improved the log - allowing inversion and updating on the fly

While its running, the "Start" button becomes a "Cancel" button.

As for the beautiful icon - I'm not much of a designer either...

[RoryB]

Can this be set to take SYS that is set up for Ervius' visual kitchen and convert it to be used for CRBuilder2?

It would take Common, specific language, specific DPI, and specific resolution folders and create a SYS folder with only the packages in it (no subfolders).

A lot of the newer SYS posts are already set up in EVK and I have not been able to get it to work for my Wizard. So I am still using CRBuilder2.

[TehPenguin]

Can this be set to take SYS that is set up for Ervius' visual kitchen and convert it to be used for CRBuilder2?

It would take Common, specific language, specific DPI, and specific resolution folders and create a SYS folder with only the packages in it (no subfolders).

A lot of the newer SYS posts are already set up in EVK and I have not been able to get it to work for my Wizard. So I am still using CRBuilder2.

So you want all of the packages in one folder? Like an unsorted dump?
Sure, no problem!

The ability to reverse sorting actually makes sense, it'd be easier then to convert from one kitchen format to another.

[RoryB]

So you want all of the packages in one folder? Like an unsorted dump?
Sure, no problem!

The ability to reverse sorting actually makes sense, it'd be easier then to convert from one kitchen format to another.
But we would need to choose which of the folder types of EVK we want to reverse into one folder. For example I do not need DPI_192 in my kitchen since I have a DPI_96 Wizard. It would be good to keep original folders and files. So I could reverse into a DPI_192 and a DPI_96 without having to do anything. Thinking that if I get a newer device I might want to create a CRBuilder2 kitchen for DPI_96 and another for DPI_192 or whatever.

[TehPenguin]

But we would need to choose which of the folder types of EVK we want to reverse into one folder. For example I do not need DPI_192 in my kitchen since I have a DPI_96 Wizard. It would be good to keep original folders and files. So I could reverse into a DPI_192 and a DPI_96 without having to do anything. Thinking that if I get a newer device I might want to create a CRBuilder2 kitchen for DPI_96 and another for DPI_192 or whatever.

So you want to be able to select which files to keep (ie Language, DPI and Res)? That shouldn't be any problem.

As for keeping the originals - this would greatly increase the time it takes to sort - better just to make a zip before your start

[kidoucorp]

Hi,

First, thanks for you great tool that save us a lot of time :D

Is there a way to just import a defined language into a new kitchen ?

For example, I have my kitchen with 040C language, and a newer dump but WWE. How to import the newer packages in my kitchen (WWE), while my 040C folders of my old kitchen's packages are copied to the newer packages ?

You could just add a function that would compare names of folders packages, not taking care of the version, and just copy missing folders from the old packages to the newer ?

That would save me a lot of time when building a non-WWE version, and updating packages btw :D

Don't know if I'm very clear of what I mean, anyway, Cheers and bigup for your work :D :rolleyes:

[Staxxthedan]

Why not just put in Zip and distribute?

[player911]

when I click the Start button, the browser just displays what looks like an XML file. I used Firefox 3.6 and IE7

[chshrm]

Great tool, thanks a lot! Now it can save me lot of time and avoid of some mistakes:D

[{SBR}_L3GION]

So you want all of the packages in one folder? Like an unsorted dump?
Sure, no problem!

The ability to reverse sorting actually makes sense, it'd be easier then to convert from one kitchen format to another.

I also would like to see this I use use evk sometimes but I have been working on migrating to OS Kitchen since its developers are still actively supporting it and when new builds get released in evk format it is time consuming to remove them all from their folders organize them in a different folder then sort to new kitchen I have been a big fan of your tool since its release and love the work I appreciate all of your work greatly.

[TehPenguin]

So you want to be able to select which files to keep (ie Language, DPI and Res)? That shouldn't be any problem.

As for keeping the originals - this would greatly increase the time it takes to sort - better just to make a zip before your start

So, after 6 months, I've finally gotten around to implementing this...

[{SBR}_L3GION]

Thank you for your hard work great release I already have it installed just awaiting a sys to try it on.

Im having Issues with the clone the donor sys folder options its not giving me a explorer box to select the folder.

[saurabh88]

love this man..awesome..
:eek::eek:

[psychocyst]

thanx a bunch man!! its a pretty awesome tool for people trying to sort the SYS packages after dumping the .dio of the latest releases (like me)
nice work helping out the xda community!!

[SGH-i200]

Im having Issues with the clone the donor sys folder options its not giving me a explorer box to select the folder.

I have the same error in PreKitchen_0.3.0.1 (same with 0.3.0.0)! But I will try on another system to check it...

[TehPenguin]

I have the same error in PreKitchen_0.3.0.1 (same with 0.3.0.0)! But I will try on another system to check it...

Sorry - my fault!
I've fixed the button in 0.3.0.2

(Also, many apologise to {SBR}_L3GION - next time feel free to PM me if I don't reply within a week)

[SGH-i200]

Sorry - my fault!
I've fixed the button in 0.3.0.2

Thanks TehPenguin! It works now!

Can you please add the ZIP to the first post?

[TehPenguin]

Thanks TehPenguin! It works now!

Can you please add the ZIP to the first post?

I though I had... I mustn't have click 'Upload'

Either way, binaries now attached to first post

[captblaze]

great tool... very useful and has not failed me yet.

one question though...

is there a way to get the default to 800 resv instead of 600, or least have a change in setting become the default?

^

[TehPenguin]

great tool... very useful and has not failed me yet.

one question though...

is there a way to get the default to 800 resv instead of 600, or least have a change in setting become the default?

^

Sorry, that was a bug on my behalf - it should be working now with 0.3.0.3

WARNING: Upgrading to 0.3.0.3 will wipe any current settings

[captblaze]

Sorry, that was a bug on my behalf - it should be working now with 0.3.0.3

WARNING: Upgrading to 0.3.0.3 will wipe any current settings

fixed... thanx

[ctiger]

Thanks for the hard work.

[kaywchan]

Thank you for developing this tool.