Search code examples
network-programmingvelocityvolumeycsbbigdata

Generating high volume and velocity data with Yahoo Cloud Serving Benchmark

YSCB offers different type of workloads.

how can i simulate real time network environment using Yahoo Cloud Serving Benchmark where:

  • we can expect communication rate can decrease or increase based on a predefined distribution
  • size of packets or records can increase or decrease based on a predefined distribution.

Basically i want to generate big data traffic using YCSB which can have different VOLUME & VELOCITY

Current YCSB offers static values for volume. These values can be changed but not varied.

The number of fields in a record

fieldcount=10

The size of each field (in bytes)

fieldlength=100

Solution

  • I managed to generate high volume by velocity by using a loop inside a loop and iterating over throughput for velocity ($j) out side and iterating field count for volume in inner loop ( $i)

    ./bin/ycsb load mongodb-async -p fieldcount=$j  -threads 4 -target $i -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0 > /home/concurrent/k