Commit graph

51 commits

Author SHA1 Message Date
janwas
24aa71b566 safely handle the case where an archive required for populating a directory cannot be opened (e.g. because it is open within 7-zip, which apparently wants exclusive access)
fixes #1021

This was SVN commit r10631.
2011-11-29 16:51:58 +00:00
Ykkrosh
7064565ff6 # Allow saving and loading single-player games.
Support writing zip files based on in-memory data.
Fix menu sliding animations to be framerate independent.

This was SVN commit r10454.
2011-10-30 00:07:28 +00:00
janwas
0d23e3f333 post-alpha sync with work.
debug stack trace fixes, remove more asm, change CONTINUE/OK scheme to
OK/ALL_COMPLETE, fix tests

This was SVN commit r9871.
2011-07-18 09:21:56 +00:00
janwas
34186dd017 refactor file interface. requires workspace update
- separate file_system_util into vfs functions (-> vfs/vfs_util) and
file_system (avoids ugly fs_util namespace prefix)
- get rid of non-portable O_BINARY_NP etc. flags
- use standard O_WRONLY etc. flags instead of LIO_WRITE; but avoid the
need for specifying O_CREAT|O_TRUNC
- only open files for aio when O_DIRECT is specified (which 0ad does
not) - avoids wasting time and security issues
- return file descriptor directly instead of via output param
- waio: safer FCB mechanism that avoids mixing descriptors between lowio
and aio

This was SVN commit r9550.
2011-05-25 10:39:13 +00:00
janwas
f608b44d1d fix self-test and map load on Windows (avoid mixing dir separators).
add more specific error codes (used at work)

This was SVN commit r9462.
2011-05-06 18:45:30 +00:00
janwas
cccd6849a7 cleanup (requires update-workspaces)
lib_errors.cpp: replace with status.cpp, adapt to needs at work
wutil: fix runtime warning reported via feedback box
config: merge CONFIG_PARANOIA and !CONFIG_FINAL into
CONFIG_ENABLE_CHECKS
add openmp, pointer_typedefs.h

This was SVN commit r9410.
2011-05-03 12:38:42 +00:00
janwas
4663ac0fe7 split debug_assert into ENSURE and ASSERT as discussed in a previous meeting.
the old debug_assert always ran and tested the expression, which slows
down release builds. wrapping them in #ifndef NDEBUG is clumsy. the new
ASSERT behaves like assert and ENSURE like the old debug_assert. Let's
change any time-critical but not-super-important ENSURE to ASSERT to
speed up release builds. (already done in bits.h and unique_range.h)

This was SVN commit r9362.
2011-04-30 13:01:45 +00:00
Ykkrosh
f22e40b64c Fix warning
This was SVN commit r9354.
2011-04-29 23:55:47 +00:00
janwas
34ba390a0d build fix for pre-C++0x compilers.
archive_zip, stream: work around possibly missing support for
std/tr1::bind in GCC by defining a functor manually
also renamed RVREF to RVALUE_REF and ensured RVALUE can convert from
both lvalue and rvalue.
io: avoid dodgy constants and possible overflow by using blockSize=0 to
indicate "don't split"

This was SVN commit r9352.
2011-04-29 22:55:35 +00:00
janwas
2374caac3e major refactor of file/io and alignment code. requires update-workspaces
. completely rewrite waio - use IOCP, add several hardcore
optimizations. now outperforms the
  AS SSD and ATTO benchmarks when writing
. refactor file interface - use LIO_READ instead of 'r', allow access to
file descriptor.
. completely rewrite the IO wrapper. now much more simple, less CPU
overhead, adds
  support for pre-issue/post-completion hooks and preallocation.
  io::Run defaults to simple synchronous IO; use io::Parameters to get
asynchronous.
. add alignment.h with constants and Align() function template (more
efficient than
  round_up for compile-time constants)
. add UniqueRange - similar to C++0x unique_ptr (emulated for C++03),
plus a
  built-in size. avoids expensive thread-safe reference counting in
shared_ptr.

cleanup:
- move fat_time functions into archive_zip
- remove no longer needed io_align and block_cache
- reduce dependencies in sysdep/compiler (move parts to
code_annotation.h)
- move IOCP into separate file (reused by waio)

This was SVN commit r9350.
2011-04-29 19:10:34 +00:00
janwas
dcd192cb60 refactor path interface:
- use wrapper class instead of std::wstring (reduces mixing of
strings/paths; allows safe+easy join via operator/ and convenient
case-insensitive comparison via operator==, avoids NativePathFromString,
similar to boost::filesystem)
- NativePath -> OsPath
- add hash and To/FromJSVal for Path
- add TS_ASSERT_PATH_EQUALS
- replace _wfopen_s with sys_OpenFile
- remove obsolete SortFiles/Directories

This was SVN commit r9107.
2011-03-23 13:36:20 +00:00
janwas
c3405e6f50 path improvements/fixes:
- replace more std::wstring with Native or VfsPath; wstring_from_utf8 ->
NativePathFromString
- replace sequences of Join(Path(), Basename+extension) with
ChangeExtension in wsdl, CacheLoader
- add Path::IsDirectory to replace .empty() / path_is_dir_sep(.back()).
also changed behavior to reflect the fact that "" is the VFS root
_directory_
- Path::Join now allows 2 identical path types (e.g. VfsPath) or one
char* literal (prevents inadvertently introducing non-safe characters).
to convert from wstring or wchar_t, use an explicit ctor (e.g.
VfsPath(wchar_t_string))

This was SVN commit r9091.
2011-03-21 19:54:08 +00:00
janwas
6d25329412 # towards locale-independent pathnames on Linux
c.f.
http://www.wildfiregames.com/forum/index.php?showtopic=14541&st=0&p=217250&#entry217250
and 2011-03-19 meeting

This was SVN commit r9090.
2011-03-21 17:53:13 +00:00
Ykkrosh
4575c26616 Disable compression of public.zip (fixes #671).
This was SVN commit r8785.
2010-12-04 15:57:55 +00:00
janwas
52851faeb6 cleanup - remove old cpu_memcpy and cpu_i64FromDouble that are no longer needed
This was SVN commit r8517.
2010-11-01 11:09:03 +00:00
Ykkrosh
a596323ea6 Avoid bogus GCC warnings about uninitialised values
This was SVN commit r8429.
2010-10-21 20:33:27 +00:00
Ykkrosh
0d172264f8 # Add -archivebuild mode to generate .zip files for releases, with automatic compression of textures.
Fix terrain manager to understand .cached.dds files.
Fix IArchiveWriter so you can pass it absolute paths of files to load,
and different relative paths for storing inside the archive.
Fix fs_util::ForEachFile when called on the VFS root.

This was SVN commit r8130.
2010-09-18 18:21:00 +00:00
janwas
c0c8132dd4 fix VFS real path retrieval for files mounted from multiple directories (e.g. mods)
closes #564
GetRealPath was too simplistic and just asked the directory *most
recently mounted* into the file's VFS directory for its path. the
correct solution is to ask the file's loader for its path. the archiver
IFileLoader new returns the archive's pathname as the real path.
also simplified VfsFile's interface.

This was SVN commit r8082.
2010-09-05 21:41:45 +00:00
Ykkrosh
1706363bab Fix #423 (Switch from CppDoc to Doxygen), based on patch from anr.
Also delete some unused declarations, and rename is_playing to
snd_is_playing.

This was SVN commit r7813.
2010-07-29 15:55:41 +00:00
janwas
1861448e6c needed to split lib into separate headers and source files, which requires lots of drudgery to specify paths for each include filename (no longer relying on same-directory lookups)
also rename wchar -> utf8 to avoid conflict with <wchar.h> (requires
rebuild of workspace)
(unfortunately copying history fails to "502 bad gateway"; had to delete
old + add new independently)

This was SVN commit r7340.
2010-03-01 14:52:58 +00:00
janwas
c2fd939898 license change to the more permissible Expat/MIT license for lib/ (see forum thread)
This was SVN commit r7316.
2010-02-08 16:23:39 +00:00
janwas
97db62c944 fix handling of non-CP1252 characters in paths
(added wide-character versions of posix opendir etc. - on Windows, we
must not convert to UTF8)

This was SVN commit r7243.
2010-01-05 19:44:30 +00:00
janwas
ff15c522fe implement changes suggested by Philip:
- add self-test
- allow decoding UTF8 values beyond BMP (and replace them later)
- quietly replace invalid bytes

This was SVN commit r7187.
2009-11-09 20:53:48 +00:00
janwas
450da0aaf7 rename [w]string_to_[w]string UTF8
add safer/more portable/less dependent on locale implementation of
wchar_t <-> UTF8

This was SVN commit r7185.
2009-11-09 14:52:51 +00:00
janwas
b48e877f5b VFS fixes+refactoring
- fix: archive_zip: avoid crash if zip file contains empty filenames
- real_directory: enable dir watch
- refactor vfs_tree interface (move recursive calls into VFS)
- VFS: return text representation instead of only printing it

This was SVN commit r7165.
2009-11-04 17:26:54 +00:00
janwas
da3030cfe6 more unicode/string cleanup:
- swprintf -> swprintf_s
- use secure_crt's _wfopen_s instead of conversion+fopen
- centralize all MBS <-> WCS conversion in new wchar.cpp (requires
workspace rebuild)
- remove no longer needed os_path
- remove unnecessary fs::wpath / VfsPath constructor casts
- fixed buffer size parameters -> ARRAY_SIZE

This was SVN commit r7162.
2009-11-03 22:27:25 +00:00
janwas
8a52113e60 huge cleanup and conversion of most string handling (especially paths) to unicode
please note: format strings must be %hs for char* arguments and %ls for
wchar_t*

This was SVN commit r7161.
2009-11-03 21:46:35 +00:00
janwas
93ab45e434 simplify file implementation (remove unnecessary layer of abstraction)
ensure struct stat has 64-bit fields on windows
also fixed some warnings

This was SVN commit r7120.
2009-08-24 21:04:01 +00:00
janwas
5b302658a3 # fix permissions of created directories
also took the opportunity to move file_system_posix to file_system since
a further FS abstraction layer = YAGNI. also namespaced
file_system_util.

This was SVN commit r7074.
2009-08-04 19:57:53 +00:00
janwas
c8cfd8b40b # groundwork for separated data/cache etc. directories (XDG)
remove path.cpp (it was based on the premise that all data files lie
below binaries/data)

This was SVN commit r7063.
2009-08-01 19:37:38 +00:00
janwas
9842b8df98 32/64-bit warning drudgery
also ensure off_t is always 64-bit (required at work)

This was SVN commit r6858.
2009-05-05 20:24:06 +00:00
Ykkrosh
1743645fd3 Simplify the overly-complex CppDoc-breaking file headers
This was SVN commit r6832.
2009-04-18 17:51:05 +00:00
Ykkrosh
214b0e6683 Removed outdated licensing information
This was SVN commit r6831.
2009-04-18 17:17:34 +00:00
Ykkrosh
c9fa7f13d9 Add GPL header
This was SVN commit r6830.
2009-04-18 17:00:33 +00:00
Ykkrosh
7a4dd7b473 Set svn:eol-style=native
This was SVN commit r6828.
2009-04-18 16:14:48 +00:00
janwas
60f48f558d # infrastructure improvements
make off_t pointer-sized (required for >4gb files at work)
add shared_ptr deleter for free(), add ASSUME_ALIGNED, warning fixes

This was SVN commit r6826.
2009-04-18 08:41:10 +00:00
janwas
16ccae10cd # fix warnings
more icc11 fixes - mostly type conversion and noncopyable (now made a
macro)

This was SVN commit r6536.
2008-12-17 16:32:46 +00:00
janwas
fe6370aff8 more dehydra. disabled lockfree module (not in use).
This was SVN commit r6240.
2008-07-17 17:00:00 +00:00
janwas
8e86d29301 dehydra fixes (mostly copy-ctor warnings)
This was SVN commit r6239.
2008-07-17 14:23:51 +00:00
Ykkrosh
504ec69508 Fixed some integer type conversion warnings.
This was SVN commit r6130.
2008-06-28 01:09:45 +00:00
janwas
cf2f61e19a fix Zip archive creation code (note: just the low-level archive code; "vfs_optimizer" is still inoperational)
archive_zip: truncate requires full path; 7-zip apparently relies on
ECDR.m_cd_numEntriesOnDisk
codec: re-enable checksum generation (both explorer and 7-z demand that
CRC be computed)
codec, codec_zlib, stream: wasn't counting the number of output bytes
generated during Finish()

This was SVN commit r6034.
2008-06-16 20:33:47 +00:00
janwas
028a92375f fixes from work:
- aligned_allocator.h: use _mm_malloc instead of _aligned_malloc
(somewhat more portable)
bits: fix incorrect handling of bit_mask(0)
- archive_zip.cpp: fix ArchiveWriter_Zip (wasn't opening its output
file, ECDR record wasn't at end of file due to padding)
- io_align.cpp: move routines to header (DLL export)
- wdbg_heap.cpp: avoid crash in report hook if CRT memory block is
invalid

This was SVN commit r6030.
2008-06-16 18:36:36 +00:00
janwas
c0ed950657 had to remove uint and ulong from lib/types.h due to conflict with other library.
this snowballed into a massive search+destroy of the hodgepodge of
mostly equivalent types we had in use (int, uint, unsigned, unsigned
int, i32, u32, ulong, uintN).

it is more efficient to use 64-bit types in 64-bit mode, so the
preferred default is size_t (for anything remotely resembling a size or
index). tile coordinates are ssize_t to allow more efficient conversion
to/from floating point. flags are int because we almost never need more
than 15 distinct bits, bit test/set is not slower and int is fastest to
type. finally, some data that is pretty much directly passed to OpenGL
is now typed accordingly.

after several hours, the code now requires fewer casts and less
guesswork.

other changes:
- unit and player IDs now have an "invalid id" constant in the
respective class to avoid casting and -1
- fix some endian/64-bit bugs in the map (un)packing. added a
convenience function to write/read a size_t.
- ia32: change CPUID interface to allow passing in ecx (required for
cache topology detection, which I need at work). remove some unneeded
functions from asm, replace with intrinsics where possible.

This was SVN commit r5942.
2008-05-11 18:48:32 +00:00
janwas
74b4ac19c1 cleanup+fixes; wdbg_heap is now feature complete.
should be safe to distribute this build

code_annotation.h: add our own noncopyable to avoid DLL base class
warning
debug: remove some unicode functions (unnecessary); minor cleanup
debug_stl: disabled
mahaf: do not warn when running as non admin
wdbg: add wdbg_printf and wdbg_assert for use in allocation hook. add
header
wdbg_heap: now feature complete.
. allows storing 3..4 frames from the call stack with 0 space overhead.
. prevents garbage file/line output.
. takes care of allocation counting (previously done by profiler)
wdbg_sym.cpp: cleanup+fixes (locking, wdbg_*)

fix: properly specify __declspec(dllimport) for certain classes

This was SVN commit r5679.
2008-02-25 21:19:52 +00:00
janwas
58360a629c - fix incorrect use of block vs. sector aligned (both offset and length must be block-aligned to allow proper caching)
- use helper functions to reduce occurrences of BLOCK_SIZE and
SECTOR_SIZE
- wmi: provide for shutting it down (gets rid of its two threads); only
initialize if actually needed (costs 200ms)
- ogl_tex: don't complain if there's no gfx_card info (due to not having
used wmi)

This was SVN commit r5572.
2008-01-20 21:52:54 +00:00
janwas
aebf8cbf68 fix VFS bug preventing directories from being created:
- vfs_lookup: currentPath was of wrong type; simplified and fix create
logic
- archive_zip: fix: correctly handle filenames with /
- vfs: remove workaround for above bug
- io: temporarily disable block caching (works around problem with zip
file created by 7z)

add and use new wdbg_heap module (WIP)
remove mmgr

This was SVN commit r5571.
2008-01-20 16:53:09 +00:00
janwas
a859562ea7 improvements and fixes:
- properly differentiate between buffer/offset alignment and length
alignment (relevant since block size has been increased to 256k)
- use VfsPath for most game paths instead of CStr
- clean up timer interface and implementation
- self-tests no longer crash
- file_cache.cpp: fix for the case where allocation fails (prevent
deleter from seeing a null pointer)
- allocators: move all shared_ptr-related stuff to its own component;
add DummySharedPtr
- codec: disable checksums (important for performance at work)
- File: made into an interface class to avoid export problems. not
entirely sure about this..
- vfs_path.h, path.h, os_path.h: proper fix for using
fs::change_extension and similar utility functions with derivatives of
basic_path
- lib_api: automatically link against import lib if building lib/ as a
DLL
- path_util: remove unused functions (this component is deprecated)
- compiler.h: add INLINE
- Xeromyces.cpp: pass PIVFS so that GetXMBPath works in self-test
(should do this mostly everywhere rather than have one singleton g_VFS)

This was SVN commit r5537.
2008-01-07 20:03:19 +00:00
janwas
8667ea74c8 fixes and improvements
- directoryPosix: replace most methods with boost filesystem (but not
all: the latter cannot efficiently enumerate files AND query their
size/mtime)
- AllocatorChecker: better name for member functions
- file: move the File class here
- trace: bugfix
- io: move UnalignedWriter to write_buffer.cpp (basically the same
thing)
- vfs: remove unnecessary "vfs" warts from variable names
- vfs_tree: VfsFile now stores single Name/Size/MTime fields instead of
the FileInfo record (less clunky)
- vfs_path: use boost filesystem's version of the basename/extension
functions
- lf_alloc: remove (no longer necessary, won't be finished - not worth
the trouble)
- path_util: remove path_foreach_component (replaced by better path
traversal logic) and PathPackage (obsoleted by fs::path)

! resource loading code now receives VfsPath as its filename. there is
also OsPath (native absolute path) and Path (relative to binaries/data)

- tex is now independent of file loading code; it just en/decodes
in-memory buffers
- wdll_ver: clean up, use smart pointer to simplify bailout code
- wsdl: remove nonexistent failure path from calc_gamma (cruised by here
because SDL_SetGamme is failing once after a cold boot at work)
- wsnd: simplify OpenAL DLL search, use boost::filesystem
- wutil: Wow64 redirection is now packaged in a (RAII) class

This was SVN commit r5525.
2007-12-22 18:15:52 +00:00
janwas
63086f4e26 part3: now functional file loading code (not entirely finalized)
(too many changes to list..)

moved tex_* files to lib/tex
archive builder and hotloading modules are disabled.\

This was SVN commit r5518.
2007-12-20 20:14:21 +00:00
janwas
75ab906315 WIP file code, getting pretty close to functional
archive: ArchiveEntry now a mememto and not exposed directly
typedefs for boost::shared_ptr
fixed up dir_util to new interface
add "FileProvider" that does Load/Store and indicates Precedence

vfs: structure has changed. one RealDirectory per OS dir; vfs_tree
stores it; populate uses it to Add() stuff to vfs_tree; lookup does path
traversal and calls populate

This was SVN commit r5499.
2007-12-01 18:32:43 +00:00