RecoverableJournalQueuingStrategy works the same way as the JournalQueuingStrategy but it extends its behaviour with recovering features. The RecoverableJournalQueuingStrategy can recover partially broken and non-commited buffers. More...

#include <RecoverableJournalQueuingStrategy.h>

Inheritance diagram for BeeeOn::RecoverableJournalQueuingStrategy:

Public Member Functions
void	setDisableTmpDataRecovery (bool disable)
	Disable running recovery of the data.tmp file.

void	setDisableBrokenRecovery (bool disable)
	Disable running recovery of broken buffers referenced from index. Such buffers are ignored (and potentially garbage collected).

void	setDisableLostRecovery (bool disable)
	Disable running recovery of lost buffers - ie. buffers that were not appended to index but contain recent data.

void	setup () override
	Setup the storage for the JournalQueuingStrategy. It creates new index or loads the existing one. All buffers present in the index are checked and registered for future use.

Public Member Functions inherited from BeeeOn::JournalQueuingStrategy
void	setRootDir (const std::string &path)
	Set the root directory where to create or use a storage.

Poco::Path	rootDir () const

void	setDisableGC (bool disable)
	Disable garbage-collection entirely. This is useful for debugging or in case of a serious bug in the GC code.

void	setNeverDropOldest (bool neverDrop)
	Disable dropping of oldest data. This is useful for debugging or in case of a serious bug in the dropping code.

void	setBytesLimit (int bytes)
	Set top limit for data consumed by the strategy in the filesystem. This counts all data files (also dangling buffers, locks and index). When setting to a negative value, it is treated as unlimited.

bool	overLimit (size_t bytes) const

void	setIgnoreIndexErrors (bool ignore)
	Configure behaviour of index loading. If the index contains broken entries, we can either fail quickly or ignore such errors and load as much as possible. Ignoring erros can lead to data loss in some cases but it is more probable that we would just peek & pop some already peeked data. The setting is applied only during JournalQueuingStrategy::setup().

bool	empty () override
	The call might call precacheEntries() to load up to 1 entry. More...

void	push (const std::vector< SensorData > &data) override
	Push the given data into a separate buffer and update the index accordingly. In case of serious failure, the method can throw exceptions.

size_t	peek (std::vector< SensorData > &data, size_t count) override
	Peek the given count of data off the registered buffers starting from the oldest one. Calling this method is stable (returns the same results) until the pop() method is called. Index is not affected by this call. More...

void	pop (size_t count) override
	Pop the count amount of data off the registered buffers. It updates index accordingly.

Protected Member Functions
void	collectRecoverable (std::list< Poco::File > &files) const
	Collect files looking like buffers that are not referenced and thus are potentially recoverable for some reason.

size_t	recoverEntries (Poco::File file, std::vector< SensorData > &data) const
	Read the given file and parse its contents like it is a buffer. In this way, we read as much of entries as possible while skipping errors. More...

std::string	recoverBrokenBuffer (Poco::File file) const
	Recover contents of the given file into a new file that has a valid digest. Empty files are deleted. If the given file represents a valid buffer with a valid digest, it is left untouched.

std::string	recoverBuffer (Poco::File tmpFile) const
	Recover the given file as it should be a buffer. This method is useful to recover file that are not named accordingly - ie. we are not sure about their validity. It calls recoverBrokenBuffer().

void	recoverTmpData (Poco::Timestamp &newest)
	Recover the data.tmp file if present and append it to the index.

void	recoverBroken (const std::string &broken, Poco::Timestamp &newest)

void	recoverLost (std::list< Poco::File > &recoverable, const Poco::Timestamp &indexModified, const Poco::Timestamp &newest)
	From a list of potentially recoverable buffers, recover those that has newer timestamps than the newest timestamp registered in the index. Also, timestamp of the last index file modification is used to speed up this process.

Protected Member Functions inherited from BeeeOn::JournalQueuingStrategy
Poco::Timestamp	initIndexAndScan (BrokenHandler broken)
	Performs all the necessary steps done when calling setup(). The given function is called when a broken buffer is detected. More...

void	initIndex (const Poco::Path &index)
	Initialize the journaling index either by creating a new empty one or by loading the existing one.

void	prescanBuffers (Poco::Timestamp &newest, BrokenHandler broken)
	Pre-scan all buffers in the index, check their consistency, collect some information (entries counts, errors, etc.) and update the index after any changes or fixes.

void	reportStats (const Poco::Timestamp &newest) const
	Report statistics about buffers.

void	inspectAndRegisterBuffer (const std::string &name, size_t offset, Poco::Timestamp &newest)
	Inspect buffer of the given name.

Journal::Ptr	index () const

std::string	writeData (const std::string &data, bool force=true) const
	Write data safely to a file and return its name.

bool	whipeFile (Poco::File file, bool usuallyFails=false) const
	Remove the given file if possible. If usuallyFails is true, than the method does not log errors.

void	collectReferenced (std::set< std::string > &referenced) const
	Collect all referenced buffers. Such buffers must not be deleted and should not be affected in anyway as they are suspects of peek()/pop() API.

bool	garbageCollect (const size_t bytes)
	Perform garbage collection to ensure that at least the given bytes amount of space is available for writing.

void	dropOldestBuffers (const size_t bytes)
	Drop oldest valid data to ensure that at least the given bytes amount of space is available for writing.

size_t	bytesUsed () const

size_t	bytesUsedAll () const

Poco::Path	pathTo (const std::string &name) const

size_t	precacheEntries (size_t count)
	The call ensures that there are up to count entries into the m_entryCache. If there is not enough data in the m_entryCache, more data is read from the appropriate buffer. This call helps to ensure that we do not read any data twice. Also, in case a buffer becomes unreadable for some reason, it would no be read again. The index is not affected by this call.

size_t	readEntries (std::function< void(const Entry &entry)> proc, size_t count)
	Read up to count entries sequentially from buffers. For each entry, call the given method proc. Buffers' offsets are being updated during this process. The method itself does not modify the index in anyway.

void	registerBuffer (const FileBuffer &buffer, const FileBufferStat &stat)
	Register the given buffer with the strategy. The index is not updated by this call.

Protected Member Functions inherited from BeeeOn::Loggable
void	setupLogger (Poco::Logger *logger=0) const

Poco::Logger &	logger () const

	Loggable (const ClassInfo &info)

	Loggable (const std::type_info &info)

Additional Inherited Members
Public Types inherited from BeeeOn::QueuingStrategy
typedef Poco::SharedPtr < QueuingStrategy >	Ptr

Protected Types inherited from BeeeOn::JournalQueuingStrategy
typedef std::function< void(const std::string &name, size_t offset, Poco::Timestamp &newest)>	BrokenHandler

Static Protected Member Functions inherited from BeeeOn::JournalQueuingStrategy
static std::string	tsString (const Poco::Timestamp &t)

Static Protected Member Functions inherited from BeeeOn::Loggable
static Poco::Logger &	forMethod (const char *name)

static Poco::Logger &	forClass (const ClassInfo &info)

static Poco::Logger &	forClass (const std::type_info &info)

template<typename T >
static Poco::Logger &	forInstance (const T *i)

static void	configureSimple (Poco::Logger &logger, const std::string &level)

static void	logException (Poco::Logger &logger, const Poco::Message::Priority priority, const Poco::Exception &e, const char *file, size_t line)

Detailed Description

RecoverableJournalQueuingStrategy works the same way as the JournalQueuingStrategy but it extends its behaviour with recovering features. The RecoverableJournalQueuingStrategy can recover partially broken and non-commited buffers.

A buffer is recovered when:

it was not commited, i.e. the data.tmp file exists, and it is non-empty
a buffer is referenced from index and has an invalid digest
an existing buffer contains timestamps newer than the newest timestamp in buffers referenced from index

In this way, we should cover the following situations:

power supply failure while writing a buffer: non-committed buffer, committed buffer not recorded in index
non-volatile media failure (written data becomes invalid)

The recovery process DOES NOT work in-situ. Buffers being recovered are first loaded into memory and such buffers are not deleted unless written back successfully.

Member Function Documentation

void RecoverableJournalQueuingStrategy::recoverBroken	(	const std::string &	broken,
		Poco::Timestamp &	newest
	)

protected

Recover a broken buffer and register it. We update the newest timestamp accordingly as the recovered buffer is valid and we have to count on it.

size_t RecoverableJournalQueuingStrategy::recoverEntries	(	Poco::File	file,
		std::vector< SensorData > &	data
	)		const

protected

Read the given file and parse its contents like it is a buffer. In this way, we read as much of entries as possible while skipping errors.

Returns: number of errors occured while parsing

The documentation for this class was generated from the following files:

/var/lib/jenkins/workspace/beeeon-gateway-master/src/exporters/RecoverableJournalQueuingStrategy.h
/var/lib/jenkins/workspace/beeeon-gateway-master/src/exporters/RecoverableJournalQueuingStrategy.cpp

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Member Function Documentation