0
\$\begingroup\$

I have the following loop which is part of a fully connected layer for a neural net:

for (int b = 0; b < batch_size; b++) {
  #pragma HLS loop_tripcount min=1 max=10
    // Output Node Iterator
    array_cpy:for (int o = 0; o < num_outputs; o++) {
    #pragma HLS loop_tripcount min=1 max=1024
      // Set bias
        small_fixedpt output_fixp;
      output_fixp = biases[o];
      //float input_sub_array[1024] = input[o*num_inputs:o*num_inputs+1024];
      small_fixedpt input_sub_array[1024] = {0};
      small_fixedpt weight_sub_array[1024] = {0};
      subcopy:for(int i = 0; i < 1024; i++) {
          input_sub_array[i] = input[b*num_inputs+i];
          weight_sub_array[i] = weights[o*num_inputs+i];
      }
      // Accumulate weighted sum
      input_loop:for (int i = 0; i < num_inputs; i++) {
      #pragma HLS loop_tripcount min=1 max=1024
          output_fixp += input_sub_array[i]*weights[i];
      }
  // Compute activation
  if(output_fixp < 0) {
      output[b*num_outputs+o] = 0;
  }
  else {
      output[b*num_outputs+o] = output_fixp;
  }
}

My input arrays are partitioned in a cyclic manner equal to the number of unroll factor in 'input_loop' which is 16.

However, this loop unrolls into a sequential manner, as seen from the analysis view:enter image description here

So i can see that my inputs are being read from seperate memory blocks. I know that i've built an accumulator but I have also tried the method of custom creating an adder tree. That does not seem to work.

I also see 16 multipliers being made (ap_fixed is the data type used):

INFO: [RTGEN 206-100] Generating core module 'fc_layer_mul_mul_sc4': 16 instance(s).



INFO: [XFORM 203-501] Unrolling loop 'subcopy' (../fc_test/fc_layer.cpp:28) in function 'fc_layer' partially with a factor of 16.
INFO: [XFORM 203-501] Unrolling loop 'input_loop' (../fc_test/fc_layer.cpp:33) in function 'fc_layer' partially with a factor of 16.
INFO: [XFORM 203-101] Partitioning array 'weights.V' (../fc_test/fc_layer.cpp:8) in dimension 1 with a cyclic factor 16.
INFO: [XFORM 203-101] Partitioning array 'input.V' (../fc_test/fc_layer.cpp:10) in dimension 1 with a cyclic factor 16.
INFO: [XFORM 203-101] Partitioning array 'input_sub_array.V' (../fc_test/fc_layer.cpp:26) in dimension 1 with a cyclic factor 16.
INFO: [XFORM 203-101] Partitioning array 'weight_sub_array.V' (../fc_test/fc_layer.cpp:27) in dimension 1 with a cyclic factor 16.

Does anyone know what's going on? This is baffling me.

Thanks

\$\endgroup\$

1 Answer 1

0
\$\begingroup\$

HLS, Sigh!

I don't see much inherent parallelism in the way you have written this, lots of nested stuff that does not have obvious parallelism.

I would be lifting your sub arrays out and having batch_size of them then the MAC could be parellelised across batch_size DSP blocks.

Also, where is weight_sub_array used? Your MAC is using the weights array directly?

You might be writing in something almost C like, but never forget you are describing hardware, not writing sequential code.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.