XRootD
XrdPfc::Info Class Reference

Status of cached file. Can be read from and written into a binary file. More...

#include <XrdPfcInfo.hh>

+ Collaboration diagram for XrdPfc::Info:

Classes

struct  AStat
 Access statistics. More...
 
struct  Status
 
union  Status.__unnamed126__
 
struct  Status.__unnamed126__.__unnamed128__
 
struct  Store
 

Public Member Functions

 Info (XrdSysTrace *trace, bool prefetchBuffer=false)
 Constructor. More...
 
 ~Info ()
 Destructor. More...
 
void CalcCksumMd5 (unsigned char *buff, char *digest)
 
uint32_t CalcCksumStore ()
 Get cksum, MD5 is for backward compatibility with V2 and V3. More...
 
uint32_t CalcCksumSyncedAndAStats ()
 
void CompactifyAccessRecords ()
 Compactify access records to the configured maximum. More...
 
int CountBlocksNotWrittenInRng (int firstIdx, int lastIdx) const
 Check download status in given block range. More...
 
void DowngradeCkSumState (CkSumCheck_e css_ref)
 
size_t GetAccessCnt () const
 Get number of accesses. More...
 
int GetBitvecSizeInBytes () const
 Get size of download-state bit-vector in bytes. More...
 
long long GetBufferSize () const
 Get prefetch buffer size. More...
 
CkSumCheck_e GetCkSumState () const
 
const char * GetCkSumStateAsText () const
 
time_t GetCreationTime () const
 Get file size. More...
 
long long GetExpectedDataFileSize () const
 Get expected data file size. More...
 
long long GetFileSize () const
 Get file size. More...
 
const AStatGetLastAccessStats () const
 Get latest access stats. More...
 
int GetLastDownloadedBlock () const
 Get number of the last downloaded block. More...
 
bool GetLatestDetachTime (time_t &t) const
 Get latest detach time. More...
 
int GetNBlocks () const
 Get number of blocks represented in download-state bit-vector. More...
 
int GetNDownloadedBlocks () const
 Get number of downloaded blocks. More...
 
long long GetNDownloadedBytes () const
 Get number of downloaded bytes. More...
 
time_t GetNoCkSumTime () const
 
time_t GetNoCkSumTimeForUVKeep () const
 
XrdSysTraceGetTrace () const
 
int GetVersion ()
 Get version. More...
 
bool HasNoCkSumTime () const
 
bool IsCkSumAny () const
 
bool IsCkSumBoth () const
 
bool IsCkSumCache () const
 
bool IsCkSumNet () const
 
bool IsComplete () const
 Get complete status. More...
 
bool Read (XrdOssDF *fp, const char *dname, const char *fname=0)
 Read content of cinfo file into this object. More...
 
const std::vector< AStat > & RefAStats () const
 
const StoreRefStoredData () const
 Get stored data. More...
 
void ResetAllAccessStats ()
 Reset IO Stats. More...
 
void ResetCkSumCache ()
 
void ResetCkSumNet ()
 
void ResetNoCkSumTime ()
 
void ResizeBits ()
 Reserve bit vectors for file_size / buffer_size bytes. More...
 
void SetAllBitsSynced ()
 Mark all blocks as synced to disk. More...
 
void SetBitPrefetch (int i)
 Mark block as obtained through prefetch. More...
 
void SetBitSynced (int i)
 Mark block as synced to disk. More...
 
void SetBitWritten (int i)
 Mark block as written to disk. More...
 
void SetBufferSizeFileSizeAndCreationTime (long long bs, long long fs)
 
void SetCkSumState (CkSumCheck_e css)
 
bool TestBitPrefetch (int i) const
 Test if block at the given index has been prefetched. More...
 
bool TestBitWritten (int i) const
 Test if block at the given index is written to disk. More...
 
void UpdateDownloadCompleteStatus ()
 Update complete status. More...
 
bool Write (XrdOssDF *fp, const char *dname, const char *fname=0)
 
void WriteIOStat (Stats &s)
 Write bytes missed, hits, and disk. More...
 
void WriteIOStatAttach ()
 Write open time in the last entry of access statistics. More...
 
void WriteIOStatDetach (Stats &s)
 Write close time together with bytes missed, hits, and disk. More...
 
void WriteIOStatSingle (long long bytes_disk)
 Write single open/close time for given bytes read from disk. More...
 
void WriteIOStatSingle (long long bytes_disk, time_t att, time_t dtc)
 Write open/close with given time and bytes read from disk. More...
 

Static Public Attributes

static const char * m_traceID = "CInfo"
 
static const int s_defaultVersion = 4
 
static const char * s_infoExtension = ".cinfo"
 
static const size_t s_infoExtensionLen = strlen(Info::s_infoExtension)
 
static size_t s_maxNumAccess = 20
 

Protected Attributes

std::vector< AStatm_astats
 access records More...
 
int m_bitvecSizeInBits
 cached More...
 
unsigned char * m_buff_prefetch
 prefetch statistics More...
 
unsigned char * m_buff_synced
 disk written state vector More...
 
unsigned char * m_buff_written
 download state vector More...
 
bool m_complete
 cached; if false, set to true when missingBlocks hit zero More...
 
bool m_hasPrefetchBuffer
 constains current prefetch score More...
 
int m_missingBlocks
 cached, updated in SetBitWritten() More...
 
Store m_store
 
XrdSysTracem_trace
 
int m_version
 

Detailed Description

Status of cached file. Can be read from and written into a binary file.

Definition at line 44 of file XrdPfcInfo.hh.


Class Documentation

◆ XrdPfc::Info::Status.__unnamed126__

union XrdPfc::Info::Status.__unnamed126__

Definition at line 48 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__:
Class Members
__unnamed126__ __unnamed__
unsigned int _raw_

◆ XrdPfc::Info::Status.__unnamed126__.__unnamed128__

struct XrdPfc::Info::Status.__unnamed126__.__unnamed128__

Definition at line 49 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__.__unnamed128__:
Class Members
int _free_bits_: 29
int f_cksum_check: 3 as in enum CkSumCheck_e

Constructor & Destructor Documentation

◆ Info()

Info::Info ( XrdSysTrace trace,
bool  prefetchBuffer = false 
)

Constructor.

Definition at line 125 of file XrdPfcInfo.cc.

125  :
126  m_trace(trace),
128  m_version(0),
130  m_missingBlocks(0),
131  m_complete(false),
132  m_hasPrefetchBuffer(prefetchBuffer),
133  m_cksCalcMd5(0)
134 {}
unsigned char * m_buff_prefetch
prefetch statistics
Definition: XrdPfcInfo.hh:326
unsigned char * m_buff_written
download state vector
Definition: XrdPfcInfo.hh:325
unsigned char * m_buff_synced
disk written state vector
Definition: XrdPfcInfo.hh:324
bool m_complete
cached; if false, set to true when missingBlocks hit zero
Definition: XrdPfcInfo.hh:332
int m_missingBlocks
cached, updated in SetBitWritten()
Definition: XrdPfcInfo.hh:331
int m_bitvecSizeInBits
cached
Definition: XrdPfcInfo.hh:330
XrdSysTrace * m_trace
Definition: XrdPfcInfo.hh:321
bool m_hasPrefetchBuffer
constains current prefetch score
Definition: XrdPfcInfo.hh:333

◆ ~Info()

Info::~Info ( )

Destructor.

Definition at line 136 of file XrdPfcInfo.cc.

137 {
138  if (m_buff_synced) free(m_buff_synced);
139  if (m_buff_written) free(m_buff_written);
140  if (m_buff_prefetch) free(m_buff_prefetch);
141  delete m_cksCalcMd5;
142 }

References m_buff_prefetch, m_buff_synced, and m_buff_written.

Member Function Documentation

◆ CalcCksumMd5()

void Info::CalcCksumMd5 ( unsigned char *  buff,
char *  digest 
)

Definition at line 240 of file XrdPfcInfo.cc.

241 {
242  if (m_cksCalcMd5)
243  m_cksCalcMd5->Init();
244  else
245  m_cksCalcMd5 = new XrdCksCalcmd5();
246 
247  m_cksCalcMd5->Update((const char*)buff, GetBitvecSizeInBytes());
248  memcpy(digest, m_cksCalcMd5->Final(), 16);
249 }
virtual char * Final()=0
virtual void Update(const char *Buff, int BLen)=0
virtual void Init()=0
int GetBitvecSizeInBytes() const
Get size of download-state bit-vector in bytes.
Definition: XrdPfcInfo.hh:433

References XrdCksCalc::Final(), GetBitvecSizeInBytes(), XrdCksCalc::Init(), and XrdCksCalc::Update().

+ Here is the call graph for this function:

◆ CalcCksumStore()

uint32_t Info::CalcCksumStore ( )

Get cksum, MD5 is for backward compatibility with V2 and V3.

Definition at line 229 of file XrdPfcInfo.cc.

230 {
231  return crc32c(0, &m_store, sizeof(Store));
232 }
uint32_t crc32c(uint32_t crc, void const *buf, size_t len)

References crc32c(), and m_store.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CalcCksumSyncedAndAStats()

uint32_t Info::CalcCksumSyncedAndAStats ( )

Definition at line 234 of file XrdPfcInfo.cc.

235 {
236  uint32_t cks = crc32c(0, m_buff_synced, GetBitvecSizeInBytes());
237  return crc32c(cks, m_astats.data(), m_astats.size() * sizeof(AStat));
238 }
std::vector< AStat > m_astats
access records
Definition: XrdPfcInfo.hh:327

References crc32c(), GetBitvecSizeInBytes(), m_astats, and m_buff_synced.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CompactifyAccessRecords()

void Info::CompactifyAccessRecords ( )

Compactify access records to the configured maximum.

Definition at line 381 of file XrdPfcInfo.cc.

382 {
383  time_t now = time(0);
384 
385  std::vector<AStat> &v = m_astats;
386 
387  for (int i = 0; i < (int) v.size() - 1; ++i)
388  {
389  if (v[i].DetachTime == 0)
390  v[i].DetachTime = std::min(v[i].AttachTime + v[i].Duration / v[i].NumIos, v[i+1].AttachTime);
391  }
392 
393  while (v.size() > s_maxNumAccess)
394  {
395  double min_s = 1e10;
396  int min_i = -1;
397 
398  int M = (int) v.size() - 2;
399  for (int i = 0; i < M; ++i)
400  {
401  AStat &a = v[i], &b = v[i + 1];
402 
403  time_t t = std::max((time_t) 1, (now - b.AttachTime) / 2 + (now - a.DetachTime) / 2);
404  double s = (double) (b.AttachTime - a.DetachTime) / t;
405 
406  if (s < min_s)
407  {
408  min_s = s;
409  min_i = i;
410  }
411  }
412  assert(min_i != -1);
413 
414  v[min_i].MergeWith(v[min_i + 1]);
415 
416  v.erase(v.begin() + (min_i + 1));
417  }
418 }
static size_t s_maxNumAccess
Definition: XrdPfcInfo.hh:315

References XrdPfc::Info::AStat::DetachTime, m_astats, and s_maxNumAccess.

Referenced by Write().

+ Here is the caller graph for this function:

◆ CountBlocksNotWrittenInRng()

int XrdPfc::Info::CountBlocksNotWrittenInRng ( int  firstIdx,
int  lastIdx 
) const
inline

Check download status in given block range.

Definition at line 456 of file XrdPfcInfo.hh.

457 {
458  // TODO rewrite to use full byte comparisons outside of edges ?
459  // Also, it seems to be always called with firstIdx = 0, lastIdx = m_bitvecSizeInBits.
460  int cnt = 0;
461  for (int i = firstIdx; i < lastIdx; ++i)
462  if (! TestBitWritten(i)) ++cnt;
463 
464  return cnt;
465 }
bool TestBitWritten(int i) const
Test if block at the given index is written to disk.
Definition: XrdPfcInfo.hh:347

References TestBitWritten().

Referenced by UpdateDownloadCompleteStatus().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ DowngradeCkSumState()

void XrdPfc::Info::DowngradeCkSumState ( CkSumCheck_e  css_ref)
inline

Definition at line 299 of file XrdPfcInfo.hh.

299 { m_store.m_status.f_cksum_check &= css_ref; }
Status m_status
status information
Definition: XrdPfcInfo.hh:87

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetAccessCnt()

size_t XrdPfc::Info::GetAccessCnt ( ) const
inline

Get number of accesses.

Definition at line 265 of file XrdPfcInfo.hh.

265 { return m_store.m_accessCnt; }
size_t m_accessCnt
total access count for the file
Definition: XrdPfcInfo.hh:86

References XrdPfc::Info::Store::m_accessCnt, and m_store.

Referenced by XrdPfc::File::GetAccessCnt().

+ Here is the caller graph for this function:

◆ GetBitvecSizeInBytes()

int XrdPfc::Info::GetBitvecSizeInBytes ( ) const
inline

Get size of download-state bit-vector in bytes.

Definition at line 433 of file XrdPfcInfo.hh.

434 {
435  if (m_bitvecSizeInBits)
436  return ((m_bitvecSizeInBits - 1)/8 + 1);
437  else
438  return 0;
439 }

References m_bitvecSizeInBits.

Referenced by CalcCksumMd5(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitPrefetch(), SetBitSynced(), SetBitWritten(), TestBitPrefetch(), TestBitWritten(), and Write().

+ Here is the caller graph for this function:

◆ GetBufferSize()

long long XrdPfc::Info::GetBufferSize ( ) const
inline

Get prefetch buffer size.

Definition at line 473 of file XrdPfcInfo.hh.

474 {
475  return m_store.m_buffer_size;
476 }
long long m_buffer_size
buffer / block size
Definition: XrdPfcInfo.hh:82

References XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::File::GetBlockSize().

+ Here is the caller graph for this function:

◆ GetCkSumState()

CkSumCheck_e XrdPfc::Info::GetCkSumState ( ) const
inline

Definition at line 290 of file XrdPfcInfo.hh.

290 { return (CkSumCheck_e) m_store.m_status.f_cksum_check; }

References XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetCkSumStateAsText()

const char * Info::GetCkSumStateAsText ( ) const

Definition at line 251 of file XrdPfcInfo.cc.

252 {
253  switch (m_store.m_status.f_cksum_check) {
254  case CSChk_None : return "none";
255  case CSChk_Cache : return "cache";
256  case CSChk_Net : return "net";
257  case CSChk_Both : return "both";
258  default : return "unknown";
259  }
260 }
@ CSChk_Both
Definition: XrdPfcTypes.hh:23
@ CSChk_Net
Definition: XrdPfcTypes.hh:23
@ CSChk_Cache
Definition: XrdPfcTypes.hh:23
@ CSChk_None
Definition: XrdPfcTypes.hh:23

References XrdPfc::CSChk_Both, XrdPfc::CSChk_Cache, XrdPfc::CSChk_Net, XrdPfc::CSChk_None, XrdPfc::Info::Store::m_status, and m_store.

◆ GetCreationTime()

time_t XrdPfc::Info::GetCreationTime ( ) const
inline

Get file size.

Definition at line 281 of file XrdPfcInfo.hh.

281 { return m_store.m_creationTime; }
time_t m_creationTime
time the info file was created
Definition: XrdPfcInfo.hh:84

References XrdPfc::Info::Store::m_creationTime, and m_store.

◆ GetExpectedDataFileSize()

long long XrdPfc::Info::GetExpectedDataFileSize ( ) const
inline

Get expected data file size.

Definition at line 424 of file XrdPfcInfo.hh.

425 {
426  int last_block = GetLastDownloadedBlock();
427  if (last_block == m_bitvecSizeInBits - 1)
428  return m_store.m_file_size;
429  else
430  return (last_block + 1) * m_store.m_buffer_size;
431 }
int GetLastDownloadedBlock() const
Get number of the last downloaded block.
Definition: XrdPfcInfo.hh:416
long long m_file_size
size of file in bytes
Definition: XrdPfcInfo.hh:83

References GetLastDownloadedBlock(), m_bitvecSizeInBits, XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_file_size, and m_store.

+ Here is the call graph for this function:

◆ GetFileSize()

long long XrdPfc::Info::GetFileSize ( ) const
inline

Get file size.

Definition at line 446 of file XrdPfcInfo.hh.

447 {
448  return m_store.m_file_size;
449 }

References XrdPfc::Info::Store::m_file_size, and m_store.

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::Cache::Stat().

+ Here is the caller graph for this function:

◆ GetLastAccessStats()

const Info::AStat * Info::GetLastAccessStats ( ) const

Get latest access stats.

Definition at line 491 of file XrdPfcInfo.cc.

492 {
493  return m_astats.empty() ? 0 : & m_astats.back();
494 }

References m_astats.

Referenced by XrdPfc::File::GetLastAccessStats().

+ Here is the caller graph for this function:

◆ GetLastDownloadedBlock()

int XrdPfc::Info::GetLastDownloadedBlock ( ) const
inline

Get number of the last downloaded block.

Definition at line 416 of file XrdPfcInfo.hh.

417 {
418  for (int i = m_bitvecSizeInBits - 1; i >= 0; --i)
419  if (TestBitWritten(i)) return i;
420 
421  return -1;
422 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by GetExpectedDataFileSize().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetLatestDetachTime()

bool Info::GetLatestDetachTime ( time_t &  t) const

Get latest detach time.

Definition at line 472 of file XrdPfcInfo.cc.

473 {
474  if (m_astats.empty())
475  {
477  }
478  else
479  {
480  const AStat& ls = m_astats.back();
481 
482  if (ls.DetachTime == 0)
483  t = ls.AttachTime + ls.Duration;
484  else
485  t = ls.DetachTime;
486  }
487 
488  return t != 0;
489 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, m_astats, XrdPfc::Info::Store::m_creationTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetNBlocks()

int XrdPfc::Info::GetNBlocks ( ) const
inline

Get number of blocks represented in download-state bit-vector.

Definition at line 441 of file XrdPfcInfo.hh.

442 {
443  return m_bitvecSizeInBits;
444 }

References m_bitvecSizeInBits.

Referenced by XrdPfc::File::GetNBlocks().

+ Here is the caller graph for this function:

◆ GetNDownloadedBlocks()

int XrdPfc::Info::GetNDownloadedBlocks ( ) const
inline

Get number of downloaded blocks.

Definition at line 402 of file XrdPfcInfo.hh.

403 {
404  int cntd = 0;
405  for (int i = 0; i < m_bitvecSizeInBits; ++i)
406  if (TestBitWritten(i)) cntd++;
407 
408  return cntd;
409 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by XrdPfc::File::GetNDownloadedBlocks(), and GetNDownloadedBytes().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNDownloadedBytes()

long long XrdPfc::Info::GetNDownloadedBytes ( ) const
inline

Get number of downloaded bytes.

Definition at line 411 of file XrdPfcInfo.hh.

412 {
414 }
int GetNDownloadedBlocks() const
Get number of downloaded blocks.
Definition: XrdPfcInfo.hh:402

References GetNDownloadedBlocks(), XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile(), and XrdPfc::Cache::ConsiderCached().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNoCkSumTime()

time_t XrdPfc::Info::GetNoCkSumTime ( ) const
inline

Definition at line 304 of file XrdPfcInfo.hh.

304 { return m_store.m_noCkSumTime; }
time_t m_noCkSumTime
time when first non-cksummed block was detected
Definition: XrdPfcInfo.hh:85

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ GetNoCkSumTimeForUVKeep()

time_t XrdPfc::Info::GetNoCkSumTimeForUVKeep ( ) const
inline

Definition at line 305 of file XrdPfcInfo.hh.

References XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetTrace()

XrdSysTrace* XrdPfc::Info::GetTrace ( ) const
inline

Definition at line 318 of file XrdPfcInfo.hh.

318 {return m_trace; }

References m_trace.

◆ GetVersion()

int XrdPfc::Info::GetVersion ( )
inline

Get version.

Definition at line 270 of file XrdPfcInfo.hh.

270 { return m_version; }

References m_version.

◆ HasNoCkSumTime()

bool XrdPfc::Info::HasNoCkSumTime ( ) const
inline

Definition at line 303 of file XrdPfcInfo.hh.

303 { return m_store.m_noCkSumTime != 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by ResetCkSumCache(), and ResetCkSumNet().

+ Here is the caller graph for this function:

◆ IsCkSumAny()

bool XrdPfc::Info::IsCkSumAny ( ) const
inline

Definition at line 295 of file XrdPfcInfo.hh.

295 { return m_store.m_status.f_cksum_check & CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumBoth()

bool XrdPfc::Info::IsCkSumBoth ( ) const
inline

Definition at line 296 of file XrdPfcInfo.hh.

296 { return (m_store.m_status.f_cksum_check & CSChk_Both) == CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumCache()

bool XrdPfc::Info::IsCkSumCache ( ) const
inline

Definition at line 293 of file XrdPfcInfo.hh.

293 { return m_store.m_status.f_cksum_check & CSChk_Cache; }

References XrdPfc::CSChk_Cache, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumCache(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsCkSumNet()

bool XrdPfc::Info::IsCkSumNet ( ) const
inline

Definition at line 294 of file XrdPfcInfo.hh.

294 { return m_store.m_status.f_cksum_check & CSChk_Net; }

References XrdPfc::CSChk_Net, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumNet(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsComplete()

bool XrdPfc::Info::IsComplete ( ) const
inline

Get complete status.

Definition at line 451 of file XrdPfcInfo.hh.

452 {
453  return m_complete;
454 }

References m_complete.

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Read(), XrdPfc::File::ReadV(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ Read()

bool Info::Read ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Read content of cinfo file into this object.

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 296 of file XrdPfcInfo.cc.

297 {
298  // Does not need lock, called only in File::Open before File::Run() starts.
299  // XXXX Wait, how about Purge, and LocalFilePath, Stat?
300 
301  TraceHeader trace_pfx("Read()", dname, fname);
302 
303  FpHelper r(fp, 0, m_trace, m_traceID, trace_pfx);
304 
305  if (r.Read(m_version)) return false;
306 
308  {
309  if (m_version == 2)
310  {
311  return ReadV2(fp, r.f_off, dname, fname);
312  }
313  else if (m_version == 3)
314  {
315  return ReadV3(fp, r.f_off, dname, fname);
316  }
317  else
318  {
319  TRACE(Warning, trace_pfx << "File version " << m_version << " not supported.");
320  return false;
321  }
322  }
323 
324  uint32_t cksum;
325 
326  if (r.Read(m_store) || r.Read(cksum)) return false;
327 
328  if (cksum != CalcCksumStore())
329  {
330  TRACE(Error, trace_pfx << "Checksum Store mismatch.");
331  return false;
332  }
333 
334  ResizeBits();
335  m_astats.resize(m_store.m_astatSize);
336 
337  if (r.ReadRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
338  r.ReadRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
339  r.Read(cksum))
340  {
341  return false;
342  }
343 
344  if (cksum != CalcCksumSyncedAndAStats())
345  {
346  TRACE(Error, trace_pfx << "Checksum Synced or AStats mismatch.");
347  return false;
348  }
349 
351 
353 
354  return true;
355 }
@ Error
#define TRACE(act, x)
Definition: XrdTrace.hh:63
void UpdateDownloadCompleteStatus()
Update complete status.
Definition: XrdPfcInfo.hh:467
uint32_t CalcCksumStore()
Get cksum, MD5 is for backward compatibility with V2 and V3.
Definition: XrdPfcInfo.cc:229
void ResizeBits()
Reserve bit vectors for file_size / buffer_size bytes.
Definition: XrdPfcInfo.cc:174
static const int s_defaultVersion
Definition: XrdPfcInfo.hh:316
uint32_t CalcCksumSyncedAndAStats()
Definition: XrdPfcInfo.cc:234
static const char * m_traceID
Definition: XrdPfcInfo.hh:312
int m_astatSize
size of AStat vector
Definition: XrdPfcInfo.hh:88

References CalcCksumStore(), CalcCksumSyncedAndAStats(), Error, GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_buff_written, m_store, m_trace, m_traceID, m_version, ResizeBits(), s_defaultVersion, TRACE, UpdateDownloadCompleteStatus(), and Macaroons::Warning.

Referenced by XrdPfc::Cache::ConsiderCached(), XrdPfc::Cache::LocalFilePath(), XrdPfc::Cache::Stat(), and XrdPfc::FPurgeState::TraverseNamespace().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ RefAStats()

const std::vector<AStat>& XrdPfc::Info::RefAStats ( ) const
inline

Definition at line 276 of file XrdPfcInfo.hh.

276 { return m_astats; }

References m_astats.

◆ RefStoredData()

const Store& XrdPfc::Info::RefStoredData ( ) const
inline

Get stored data.

Definition at line 275 of file XrdPfcInfo.hh.

275 { return m_store; }

References m_store.

◆ ResetAllAccessStats()

void Info::ResetAllAccessStats ( )

Reset IO Stats.

Definition at line 361 of file XrdPfcInfo.cc.

362 {
363  m_store.m_accessCnt = 0;
364  m_store.m_astatSize = 0;
365  m_astats.clear();
366 }

References XrdPfc::Info::Store::m_accessCnt, m_astats, XrdPfc::Info::Store::m_astatSize, and m_store.

◆ ResetCkSumCache()

void Info::ResetCkSumCache ( )

Definition at line 205 of file XrdPfcInfo.cc.

206 {
207  if (IsCkSumCache())
208  {
209  m_store.m_status.f_cksum_check &= ~CSChk_Cache;
210  if ( ! HasNoCkSumTime())
211  m_store.m_noCkSumTime = time(0);
212  }
213 }
bool IsCkSumCache() const
Definition: XrdPfcInfo.hh:293
bool HasNoCkSumTime() const
Definition: XrdPfcInfo.hh:303

References XrdPfc::CSChk_Cache, HasNoCkSumTime(), IsCkSumCache(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

+ Here is the call graph for this function:

◆ ResetCkSumNet()

void Info::ResetCkSumNet ( )

Definition at line 215 of file XrdPfcInfo.cc.

216 {
217  if (IsCkSumNet())
218  {
219  m_store.m_status.f_cksum_check &= ~CSChk_Net;
220  if ( ! HasNoCkSumTime())
221  m_store.m_noCkSumTime = time(0);
222  }
223 }
bool IsCkSumNet() const
Definition: XrdPfcInfo.hh:294

References XrdPfc::CSChk_Net, HasNoCkSumTime(), IsCkSumNet(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ResetNoCkSumTime()

void XrdPfc::Info::ResetNoCkSumTime ( )
inline

Definition at line 306 of file XrdPfcInfo.hh.

306 { m_store.m_noCkSumTime = 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ ResizeBits()

void Info::ResizeBits ( )

Reserve bit vectors for file_size / buffer_size bytes.

Definition at line 174 of file XrdPfcInfo.cc.

175 {
176  // drop buffer in case of failed/partial reads
177 
178  if (m_buff_synced) free(m_buff_synced);
179  if (m_buff_written) free(m_buff_written);
180  if (m_buff_prefetch) free(m_buff_prefetch);
181 
183 
184  m_buff_written = (unsigned char*) malloc(GetBitvecSizeInBytes());
185  m_buff_synced = (unsigned char*) malloc(GetBitvecSizeInBytes());
186  memset(m_buff_written, 0, GetBitvecSizeInBytes());
187  memset(m_buff_synced, 0, GetBitvecSizeInBytes());
188 
190  m_complete = false;
191 
193  {
194  m_buff_prefetch = (unsigned char*) malloc(GetBitvecSizeInBytes());
196  }
197  else
198  {
199  m_buff_prefetch = 0;
200  }
201 }

References GetBitvecSizeInBytes(), m_bitvecSizeInBits, m_buff_prefetch, m_buff_synced, m_buff_written, XrdPfc::Info::Store::m_buffer_size, m_complete, XrdPfc::Info::Store::m_file_size, m_hasPrefetchBuffer, m_missingBlocks, and m_store.

Referenced by Read(), and SetBufferSizeFileSizeAndCreationTime().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetAllBitsSynced()

void Info::SetAllBitsSynced ( )

Mark all blocks as synced to disk.

Definition at line 146 of file XrdPfcInfo.cc.

147 {
148  // The following should be:
149  // memset(m_buff_synced, 255, GetBitvecSizeInBytes());
150  // but GCC produces an overzealous 'possible argument transpose warning' and
151  // xrootd build uses warnings->errors escalation.
152  // This workaround can be removed for gcc >= 5.
153  // See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61294
154  const int nb = GetBitvecSizeInBytes();
155  for (int i = 0; i < nb; ++i)
156  m_buff_synced[i] = 255;
157 
158  m_complete = true;
159 }

References GetBitvecSizeInBytes(), m_buff_synced, and m_complete.

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitPrefetch()

void XrdPfc::Info::SetBitPrefetch ( int  i)
inline

Mark block as obtained through prefetch.

Definition at line 369 of file XrdPfcInfo.hh.

370 {
371  if (!m_buff_prefetch) return;
372 
373  const int cn = i/8;
374  assert(cn < GetBitvecSizeInBytes());
375 
376  const int off = i - cn*8;
377  m_buff_prefetch[cn] |= cfiBIT(off);
378 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitSynced()

void XrdPfc::Info::SetBitSynced ( int  i)
inline

Mark block as synced to disk.

Definition at line 391 of file XrdPfcInfo.hh.

392 {
393  const int cn = i/8;
394  assert(cn < GetBitvecSizeInBytes());
395 
396  const int off = i - cn*8;
397  m_buff_synced[cn] |= cfiBIT(off);
398 }

References GetBitvecSizeInBytes(), and m_buff_synced.

Referenced by XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitWritten()

void XrdPfc::Info::SetBitWritten ( int  i)
inline

Mark block as written to disk.

Definition at line 356 of file XrdPfcInfo.hh.

357 {
358  const int cn = i/8;
359  assert(cn < GetBitvecSizeInBytes());
360 
361  const int off = i - cn*8;
362 
363  m_buff_written[cn] |= cfiBIT(off);
364 
365  if (--m_missingBlocks == 0)
366  m_complete = true;
367 }

References GetBitvecSizeInBytes(), m_buff_written, m_complete, and m_missingBlocks.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBufferSizeFileSizeAndCreationTime()

void Info::SetBufferSizeFileSizeAndCreationTime ( long long  bs,
long long  fs 
)

Definition at line 163 of file XrdPfcInfo.cc.

164 {
165  // Needed only when Info object is created for the first time in File::Open()
166  m_store.m_buffer_size = bs;
167  m_store.m_file_size = fs;
168  ResizeBits();
169  m_store.m_creationTime = time(0);
170 }

References XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_file_size, m_store, and ResizeBits().

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetCkSumState()

void XrdPfc::Info::SetCkSumState ( CkSumCheck_e  css)
inline

Definition at line 298 of file XrdPfcInfo.hh.

298 { m_store.m_status.f_cksum_check = css; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ TestBitPrefetch()

bool XrdPfc::Info::TestBitPrefetch ( int  i) const
inline

Test if block at the given index has been prefetched.

Definition at line 380 of file XrdPfcInfo.hh.

381 {
382  if (!m_buff_prefetch) return false;
383 
384  const int cn = i/8;
385  assert(cn < GetBitvecSizeInBytes());
386 
387  const int off = i - cn*8;
388  return (m_buff_prefetch[cn] & cfiBIT(off)) != 0;
389 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

+ Here is the call graph for this function:

◆ TestBitWritten()

bool XrdPfc::Info::TestBitWritten ( int  i) const
inline

Test if block at the given index is written to disk.

Definition at line 347 of file XrdPfcInfo.hh.

348 {
349  const int cn = i/8;
350  assert(cn < GetBitvecSizeInBytes());
351 
352  const int off = i - cn*8;
353  return (m_buff_written[cn] & cfiBIT(off)) != 0;
354 }

References GetBitvecSizeInBytes(), and m_buff_written.

Referenced by CountBlocksNotWrittenInRng(), GetLastDownloadedBlock(), GetNDownloadedBlocks(), and XrdPfc::File::Prefetch().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ UpdateDownloadCompleteStatus()

void XrdPfc::Info::UpdateDownloadCompleteStatus ( )
inline

Update complete status.

Definition at line 467 of file XrdPfcInfo.hh.

468 {
470  m_complete = (m_missingBlocks == 0);
471 }
int CountBlocksNotWrittenInRng(int firstIdx, int lastIdx) const
Check download status in given block range.
Definition: XrdPfcInfo.hh:456

References CountBlocksNotWrittenInRng(), m_bitvecSizeInBits, m_complete, and m_missingBlocks.

Referenced by Read().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ Write()

bool Info::Write ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Write number of blocks and read buffer size

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 268 of file XrdPfcInfo.cc.

269 {
270  TraceHeader trace_pfx("Write()", dname, fname);
271 
273  m_store.m_astatSize = (int32_t) m_astats.size();
274 
275  FpHelper w(fp, 0, m_trace, m_traceID, trace_pfx);
276 
277  if (w.Write(s_defaultVersion) ||
278  w.Write(m_store) ||
279  w.Write(CalcCksumStore()) ||
280  w.WriteRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
281  w.WriteRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
282  w.Write(CalcCksumSyncedAndAStats()))
283  {
284  return false;
285  }
286 
287  return true;
288 }
void CompactifyAccessRecords()
Compactify access records to the configured maximum.
Definition: XrdPfcInfo.cc:381

References CalcCksumStore(), CalcCksumSyncedAndAStats(), CompactifyAccessRecords(), GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_store, m_trace, m_traceID, s_defaultVersion, and s_maxNumAccess.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::File::Sync().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStat()

void Info::WriteIOStat ( Stats s)

Write bytes missed, hits, and disk.

Definition at line 431 of file XrdPfcInfo.cc.

432 {
433  m_astats.back().NumIos = s.m_NumIos;
434  m_astats.back().Duration = s.m_Duration;
435  m_astats.back().BytesHit = s.m_BytesHit;
436  m_astats.back().BytesMissed = s.m_BytesMissed;
437  m_astats.back().BytesBypassed = s.m_BytesBypassed;
438 }
long long m_BytesMissed
number of bytes served from remote and cached
Definition: XrdPfcStats.hh:36
long long m_BytesBypassed
number of bytes served directly through XrdCl
Definition: XrdPfcStats.hh:37
int m_Duration
total duration of all IOs attached
Definition: XrdPfcStats.hh:34
int m_NumIos
number of IO objects attached during this access
Definition: XrdPfcStats.hh:33
long long m_BytesHit
number of bytes served from disk
Definition: XrdPfcStats.hh:35

References m_astats, XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_Duration, and XrdPfc::Stats::m_NumIos.

Referenced by XrdPfc::File::Sync(), and WriteIOStatDetach().

+ Here is the caller graph for this function:

◆ WriteIOStatAttach()

void Info::WriteIOStatAttach ( )

Write open time in the last entry of access statistics.

Definition at line 422 of file XrdPfcInfo.cc.

423 {
425 
426  AStat as;
427  as.AttachTime = time(0);
428  m_astats.push_back(as);
429 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, and m_store.

◆ WriteIOStatDetach()

void Info::WriteIOStatDetach ( Stats s)

Write close time together with bytes missed, hits, and disk.

Definition at line 440 of file XrdPfcInfo.cc.

441 {
442  m_astats.back().DetachTime = time(0);
443  WriteIOStat(s);
444 }
void WriteIOStat(Stats &s)
Write bytes missed, hits, and disk.
Definition: XrdPfcInfo.cc:431

References m_astats, and WriteIOStat().

Referenced by XrdPfc::File::FinalizeSyncBeforeExit().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [1/2]

void Info::WriteIOStatSingle ( long long  bytes_disk)

Write single open/close time for given bytes read from disk.

Definition at line 446 of file XrdPfcInfo.cc.

447 {
449 
450  AStat as;
451  as.AttachTime = as.DetachTime = time(0);
452  as.NumIos = 1;
453  as.BytesHit = bytes_disk;
454  m_astats.push_back(as);
455 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [2/2]

void Info::WriteIOStatSingle ( long long  bytes_disk,
time_t  att,
time_t  dtc 
)

Write open/close with given time and bytes read from disk.

Definition at line 457 of file XrdPfcInfo.cc.

458 {
460 
461  AStat as;
462  as.AttachTime = att;
463  as.DetachTime = dtc;
464  as.NumIos = 1;
465  as.Duration = dtc - att;
466  as.BytesHit = bytes_disk;
467  m_astats.push_back(as);
468 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Member Data Documentation

◆ m_astats

◆ m_bitvecSizeInBits

int XrdPfc::Info::m_bitvecSizeInBits
protected

◆ m_buff_prefetch

unsigned char* XrdPfc::Info::m_buff_prefetch
protected

prefetch statistics

Definition at line 326 of file XrdPfcInfo.hh.

Referenced by ~Info(), ResizeBits(), SetBitPrefetch(), and TestBitPrefetch().

◆ m_buff_synced

unsigned char* XrdPfc::Info::m_buff_synced
protected

disk written state vector

Definition at line 324 of file XrdPfcInfo.hh.

Referenced by ~Info(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitSynced(), and Write().

◆ m_buff_written

unsigned char* XrdPfc::Info::m_buff_written
protected

download state vector

Definition at line 325 of file XrdPfcInfo.hh.

Referenced by ~Info(), Read(), ResizeBits(), SetBitWritten(), and TestBitWritten().

◆ m_complete

bool XrdPfc::Info::m_complete
protected

cached; if false, set to true when missingBlocks hit zero

Definition at line 332 of file XrdPfcInfo.hh.

Referenced by IsComplete(), ResizeBits(), SetAllBitsSynced(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_hasPrefetchBuffer

bool XrdPfc::Info::m_hasPrefetchBuffer
protected

constains current prefetch score

Definition at line 333 of file XrdPfcInfo.hh.

Referenced by ResizeBits().

◆ m_missingBlocks

int XrdPfc::Info::m_missingBlocks
protected

cached, updated in SetBitWritten()

Definition at line 331 of file XrdPfcInfo.hh.

Referenced by ResizeBits(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_store

◆ m_trace

XrdSysTrace* XrdPfc::Info::m_trace
protected

Definition at line 321 of file XrdPfcInfo.hh.

Referenced by GetTrace(), Read(), and Write().

◆ m_traceID

const char * Info::m_traceID = "CInfo"
static

Definition at line 312 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ m_version

int XrdPfc::Info::m_version
protected

Definition at line 329 of file XrdPfcInfo.hh.

Referenced by GetVersion(), and Read().

◆ s_defaultVersion

const int Info::s_defaultVersion = 4
static

Definition at line 316 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ s_infoExtension

◆ s_infoExtensionLen

const size_t Info::s_infoExtensionLen = strlen(Info::s_infoExtension)
static

Definition at line 314 of file XrdPfcInfo.hh.

◆ s_maxNumAccess

size_t Info::s_maxNumAccess = 20
static

Definition at line 315 of file XrdPfcInfo.hh.

Referenced by CompactifyAccessRecords(), XrdPfc::Cache::Config(), and Write().


The documentation for this class was generated from the following files: