Skip to content

Commit b733fe6

Browse files
committed
Should close #13
Cleaned up the implementation a little, I'm sure it can still be optimized further. Undid some hacky things in rapidjson that the old implementation utilized.
1 parent 8550026 commit b733fe6

File tree

3 files changed

+63
-47
lines changed

3 files changed

+63
-47
lines changed

include/cereal/archives/json.hpp

Lines changed: 61 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -279,9 +279,32 @@ namespace cereal
279279
Input JSON should have been produced by the JSONOutputArchive. Data can
280280
only be added to dynamically sized containers (marked by JSON arrays) -
281281
the input archive will determine their size by looking at the number of child nodes.
282-
283-
The order of the items in the JSON archive must match what is expected in the
284-
serialization functions.
282+
Only JSON originating from a JSONOutputArchive is officially supported, but data
283+
from other sources may work if properly formatted.
284+
285+
The JSONInputArchive does not require that nodes are loaded in the same
286+
order they were saved by JSONOutputArchive. Using name value pairs (NVPs),
287+
it is possible to load in an out of order fashion or otherwise skip/select
288+
specific nodes to load.
289+
290+
The default behavior of the input archive is to read sequentially starting
291+
with the first node and exploring its children. When a given NVP does
292+
not match the read in name for a node, the archive will search for that
293+
node at the current level and load it if it exists. After loading an out of
294+
order node, the archive will then proceed back to loading sequentially from
295+
its new position.
296+
297+
Consider this simple example where loading of some data is skipped:
298+
299+
@code{cpp}
300+
// imagine the input file has someData(1-9) saved in order at the top level node
301+
ar( someData1, someData2, someData3 ); // XML loads in the order it sees in the file
302+
ar( cereal::make_nvp( "hello", someData6 ) ); // NVP given does not
303+
// match expected NVP name, so we search
304+
// for the given NVP and load that value
305+
ar( someData7, someData8, someData9 ); // with no NVP given, loading resumes at its
306+
// current location, proceeding sequentially
307+
@endcode
285308
286309
\ingroup Archives */
287310
class JSONInputArchive : public InputArchive<JSONInputArchive>
@@ -344,25 +367,20 @@ namespace cereal
344367
class Iterator
345368
{
346369
public:
347-
Iterator() : itsType(Null) {}
370+
Iterator() : itsIndex( 0 ), itsType(Null) {}
348371

349372
Iterator(MemberIterator begin, MemberIterator end) :
350-
itsMemberIt(begin), itsMemberItEnd(end), itsType(Member)
373+
itsMemberItBegin(begin), itsMemberItEnd(end), itsIndex(0), itsType(Member)
351374
{ }
352375

353376
Iterator(ValueIterator begin, ValueIterator end) :
354-
itsValueIt(begin), itsValueItEnd(end), itsType(Value)
377+
itsValueItBegin(begin), itsValueItEnd(end), itsIndex(0), itsType(Value)
355378
{ }
356379

357380
//! Advance to the next node
358381
Iterator & operator++()
359382
{
360-
switch(itsType)
361-
{
362-
case Value : ++itsValueIt; break;
363-
case Member: /*std::cerr << "Advancing from " << name() << std::endl;*/ ++itsMemberIt; break;
364-
default: throw cereal::Exception("Invalid Iterator Type!");
365-
}
383+
++itsIndex;
366384
return *this;
367385
}
368386

@@ -371,63 +389,63 @@ namespace cereal
371389
{
372390
switch(itsType)
373391
{
374-
case Value : return *itsValueIt;
375-
case Member: return itsMemberIt->value;
392+
case Value : return itsValueItBegin[itsIndex];
393+
case Member: return itsMemberItBegin[itsIndex].value;
376394
default: throw cereal::Exception("Invalid Iterator Type!");
377395
}
378396
}
379397

380398
//! Get the name of the current node, or nullptr if it has no name
381399
const char * name() const
382400
{
383-
if( itsType == Member && itsMemberIt != itsMemberItEnd )
384-
return itsMemberIt->name.GetString();
401+
if( itsType == Member && (itsMemberItBegin + itsIndex) != itsMemberItEnd )
402+
return itsMemberItBegin[itsIndex].name.GetString();
385403
else
386404
return nullptr;
387405
}
388406

389407
//! Adjust our position such that we are at the node with the given name
390408
/*! @throws Exception if no such named node exists */
391-
inline void search( const char * name, GenericValue const & parent )
409+
inline void search( const char * name )//, GenericValue const & parent )
392410
{
393-
auto member = parent.FindMember( name );
394-
if( member )
395-
itsMemberIt = member;
396-
else
397-
throw Exception("JSON Parsing failed - provided NVP not found");
411+
size_t index = 0;
412+
for( auto it = itsMemberItBegin; it != itsMemberItEnd; ++it, ++index )
413+
if( std::strcmp( name, it->name.GetString() ) == 0 )
414+
{
415+
itsIndex = index;
416+
return;
417+
}
418+
419+
throw Exception("JSON Parsing failed - provided NVP not found");
398420
}
399421

400422
private:
401-
MemberIterator itsMemberIt, itsMemberItEnd; //!< The member iterator (object)
402-
ValueIterator itsValueIt, itsValueItEnd; //!< The value iterator (array)
423+
MemberIterator itsMemberItBegin, itsMemberItEnd; //!< The member iterator (object)
424+
ValueIterator itsValueItBegin, itsValueItEnd; //!< The value iterator (array)
425+
size_t itsIndex; //!< The current index of this iterator
403426
enum Type {Value, Member, Null} itsType; //!< Whether this holds values (array) or members (objects) or nothing
404427
};
405428

406429
//! Searches for the expectedName node if it doesn't match the actualName
407-
/*! @throws Exception if an expectedName is given and not found */
430+
/*! This needs to be called before every load or node start occurs. This function will
431+
check to see if an NVP has been provided (with setNextName) and if so, see if that name matches the actual
432+
next name given. If the names do not match, it will search in the current level of the JSON for that name.
433+
If the name is not found, an exception will be thrown.
434+
435+
Resets the NVP name after called.
436+
437+
@throws Exception if an expectedName is given and not found */
408438
inline void search()
409439
{
410440
// The name an NVP provided with setNextName()
411441
if( itsNextName )
412442
{
413-
//std::cerr << "Next name is " << itsNextName << std::endl;
414-
//std::cerr << itsIteratorStack.size() << std::endl;
415443
// The actual name of the current node
416444
auto const actualName = itsIteratorStack.back().name();
417445

418-
//std::cerr << "Actual name was: " << (actualName?actualName:"null") << std::endl;
419-
// when at end of an iterator, the next name will be null
420-
421-
if( itsIteratorStack.back().value().IsNull() || !actualName || std::strcmp( itsNextName, actualName ) != 0 )
422-
{
423-
//std::cerr << "Searching for " << itsNextName << std::endl;
424-
//std::cerr << itsIteratorStack.size() << std::endl;
425-
// names don't match, perform a search and adjust our current iterator
426-
itsIteratorStack.back().search( itsNextName,
427-
/*if*/ (itsIteratorStack.size() > 1 ?
428-
/*then*/ (itsIteratorStack.rbegin() + 1)->value() :
429-
/*else*/ itsDocument ) );
430-
}
446+
// Do a search if we don't see a name coming up, or if the names don't match
447+
if( !actualName || std::strcmp( itsNextName, actualName ) != 0 )
448+
itsIteratorStack.back().search( itsNextName );
431449
}
432450

433451
itsNextName = nullptr;
@@ -438,13 +456,12 @@ namespace cereal
438456
/*! This places an iterator for the next node to be parsed onto the iterator stack. If the next
439457
node is an array, this will be a value iterator, otherwise it will be a member iterator.
440458
441-
By default our strategy is to start with the document root node and then recursively iteratoe through
459+
By default our strategy is to start with the document root node and then recursively iterate through
442460
all children in the order they show up in the document.
443461
We don't need to know NVPs to do this; we'll just blindly load in the order things appear in.
444462
445-
We check to see if the specified NVP matches what the next automatically loaded node is. If they
446-
match, we just continue as normal, going in order. If they don't match, we attempt to find a node
447-
named after the NVP that is being loaded. If that NVP does not exist, we throw an exception */
463+
If we were given an NVP, we will search for it if it does not match our the name of the next node
464+
that would normally be loaded. This functionality is provided by search(). */
448465
void startNode()
449466
{
450467
search();
@@ -460,7 +477,6 @@ namespace cereal
460477
{
461478
itsIteratorStack.pop_back();
462479
++itsIteratorStack.back();
463-
//std::cerr << "Finishing a node " << itsIteratorStack.size() << std::endl;
464480
}
465481

466482
//! Sets the name for the next node created with startNode

include/cereal/external/rapidjson/document.h

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -614,7 +614,6 @@ int z = a[0u].GetInt(); // This works too.
614614
Array a;
615615
}; // 12 bytes in 32-bit mode, 16 bytes in 64-bit mode
616616

617-
public:
618617
//! Find member by name.
619618
Member* FindMember(const Ch* name) {
620619
RAPIDJSON_ASSERT(name);
@@ -630,7 +629,6 @@ int z = a[0u].GetInt(); // This works too.
630629
return 0;
631630
}
632631
const Member* FindMember(const Ch* name) const { return const_cast<GenericValue&>(*this).FindMember(name); }
633-
private:
634632

635633
// Initialize this value as array with initial data, without calling destructor.
636634
void SetArrayRaw(GenericValue* values, SizeType count, Allocator& alloctaor) {

sandbox.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -583,6 +583,8 @@ int main()
583583

584584
oar( bb, a, x, y, z, d, j );
585585
std::cout << bb << " " << a << " " << x << " " << y << " " << z << " " << d << " " << j << std::endl;
586+
// valgrind will complain about uninitialized bytes here - seems to be the padding caused by the long double and
587+
// long long allocations (this padding just exists on the stack and is never used anywhere)
586588
}
587589
{
588590
std::ifstream b("endian.out");

0 commit comments

Comments
 (0)