Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2 3

Author Topic: dfWorldDat : the Dwarf Fortress World.dat File Format Project  (Read 19129 times)

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile

Dwarf Fortress World.dat File Format for df34.07
dfWorldDat current version: 0

Link to dfWorldDat SourceForge Project

Primary Goal:
To provide synchronization between a World.dat or World.sav and it's raws, so that modders may quickly test changes without re-genning a new world.
(in other words, replace DF World Tinker)

Secondary Goal:
To provide an export mechanism giving world information so that third party GUIs may be made to render the world, view it in detail, search it's layers for ideal embark sites, generate movies of world history (oh yeah), etc, etc.

About:
I've been working on this since the early releases of df2010, extending as needed when needed.
Getting burned out deciphering the cryptic world format, and too busy to pursue time consuming trial-and-error discovery at the moment.
So I cleaned up the file, commented it, and opened up what I have to create a team project for it.
The SF project has one file, a template for the 010 Hex Editor by Sweetscape.
This will parse most of the header, RAW tags, geology data, regions and some subregions.  Then it will error out on a subregion.
I need help completing this, by people with a reasonable understanding of hex editing, data structures, grok 'little-endian', basic C knowledge.
And most importantly, people willing to do trial-and-error experimentation to discover the inner workings of the World.dat format.

Tasks:
Listed roughly in the order found in the world.dat file, difficulty varies.
Spoiler (click to show/hide)

Getting Started:
  • Turn off compressed saves in init.txt, and use vanilla RAWs
  • Get 010 Hex Editor by Sweetscape
  • Get the latest DwarfFortress3407Template.bt from the dfWorldDat SF project.
  • Read the 010 Editor help files about scripts and templates.
  • Gen a generic pocket world, open the world.dat file in 010, load the template, run it.
  • It will fail somewhere in the subregions.  Take a look around, get a feel for it.
  • Now is a good time to skim through the whole template file, to understand what it is doing.
  • Dive in.

I've documented the template file fairly thoroughly.
Some things I learned with dfWorldTinker have not been incorporated yet, but not much.
If you want to contribute, come back to this page to see what tasks have been assigned, then pick one still open and PM me that you're working on it.
(All of them are open right now.)
If there is enough interest, I'll get the SourceForge page better organized.
If there is a very large amount of interest, I'll move over to GitHub.

« Last Edit: April 20, 2012, 09:46:10 am by zilpin »
Logged

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #1 on: June 09, 2011, 10:54:54 pm »

(placeholder)
Logged

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #2 on: June 09, 2011, 10:55:35 pm »

(placeholder)
Logged

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #3 on: June 09, 2011, 10:55:54 pm »

(placeholder)
Logged

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #4 on: June 09, 2011, 11:05:28 pm »


One last note, back before civ sprawl was implemented, I was able to read the Site data as well (caves, cities, towers).  Some of the complications with subregions may be due to cites spanning multiple embark regions, farms, and all that was added later being stored in the subregion data.
Worth thinking about, but until you've gotten into viewing the data in hex, that probably does not mean much to you.
Logged

Alem

  • Bay Watcher
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #5 on: June 10, 2011, 02:22:44 am »

I heartily approve of this concept. I have zero skills to help you, but I highly approve. World Tinker is handy, but feels a little on the temperamental side currently, and apparently found at least one of my fortresses delicious :)
Logged

Grimlocke

  • Bay Watcher
  • *kobold noises*
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #6 on: June 15, 2011, 09:19:49 am »

Not sure how I missed this, but following it now. This could save loads of time with testing mods, and save quite a few of those fortresses I end up abandoning due to some fault in my attempts at modding.

Sadly best I can do to help you is to cheers you on from the sidelines, as I am programming language illiterate.
Logged
I make Grimlocke's History & Realism Mods. Its got poleaxes, sturdy joints and bloomeries. Now compatible with DF Revised!

Thundercraft

  • Bay Watcher
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #7 on: June 15, 2011, 11:38:12 am »

Posting to follow this thread.
Logged

darius

  • Bay Watcher
  • ^^
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #8 on: June 15, 2011, 05:00:58 pm »

Pls add power goal: ability to synchronize two (and more) different braches of same world (reassigning id's of creatures and artifact items) so that it could be synced between many players thus forming an indirect multiplayer mode (e.g. i killed something so update server that he is dead, and tell legends about my character and so on)
Powergoal because hard to achieve ( almost everything must be found out) and because it would be awesome
Logged

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #9 on: June 15, 2011, 07:13:20 pm »

All goals at this point will be in reading and understanding the file format.
This is a discovery project, at the moment.  After enough is understood about the file format, an API will be designed.  Then a command line program using it. 
If sufficient people step up to help with the trial and error hex editing.

So far, no one has.

[edit]
Or, if the Almighty Toad smiles upon us, and tells us what the file format is.
Savegame reading/writing code, even without required header files, would have no risk on his part and save us a great deal of time.
Not likely, but it will be in my prayers to Armok to speak favorably with his favored son  :P
« Last Edit: June 15, 2011, 07:36:41 pm by zilpin »
Logged

Artanis00

  • Bay Watcher
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #10 on: June 15, 2011, 09:32:03 pm »

Or, if the Almighty Toad smiles upon us, and tells us what the file format is.
Savegame reading/writing code, even without required header files, would have no risk on his part and save us a great deal of time.
Not likely, but it will be in my prayers to Armok to speak favorably with his favored son  :P

I believe his no API stance extends (or would extend) to this kind of thing, though slightly mitigated by the fact that the save format isn't as flexible as internals (he has to consider backward compatibility and all).
Logged
Git - fast, efficient, distributed version control system
Github - Free public repositories, issue tracking, wikis, downloads...

zilpin

  • Bay Watcher
  • 437 forever!
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #11 on: September 22, 2011, 12:29:08 pm »


Several months, but with only 32 downloads of the template file, and no one volunteering, this is not going anywhere right now.

Anybody who can contribute, need you.

Logged

Dienes

  • Bay Watcher
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #12 on: September 23, 2011, 12:50:07 am »

Oh golly this looks neat. There was a mad plan discussed that could be very fun but would need to know what was going on in world.dat to happen. I did some looking but must have missed this so I had mostly given up in despair.

My hex editing is rusty but I should have some time next week to see if I can made some sense of this.

Prayers and catnip for Scamps will probably pay off before Armok.
Logged

Andux

  • Bay Watcher
  • [PREFSTRING:semicolons]
    • View Profile
    • Andux's DFWiki page
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #13 on: September 23, 2011, 08:06:03 am »

I did some work on documenting DF's file formats earlier this year; never got very far, but maybe it can help fill in some gaps:

http://df.magmawiki.com/index.php/User:Andux/Format_research
Logged
(Do not sign anything.) -- Fell, Planescape: Torment

MADMAN · Save Tools · WTF Tools · Generated Raws Extractor · Tweak for 0.31–34.xx

Ghills

  • Bay Watcher
    • View Profile
Re: dfWorldDat : the Dwarf Fortress World.dat File Format Project
« Reply #14 on: September 23, 2011, 08:23:33 pm »

The hex editor program you recommend is only free for 30 days.  Does your program need that hex editor, or would another one work?
Logged
I AM POINTY DEATH INCARNATE
Ye know, being an usurper overseer gone mad with power isn't too bad. It's honestly not that different from being a normal overseer.
To summarize:
They do an epic face. If that fails, they beat said object to death with their beard.
Pages: [1] 2 3