Re: Performance (was Re: Selenophobia)
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Re: Performance (was Re: Selenophobia)
- From: Frank Kastenholz <fkastenholz@...>
- Date: 2017年3月26日 10:35:48 -0400
On 3/25/17 4:48 PM, Tim Hill wrote:
In our project, we avoided the “Lua is too slow” issue by providing a
robust set of fast C libraries that did all the heavy lifting, leaving
Lua to do business logic and decision making. I presented it as “C =
muscle, Lua = brain”, which seemed to work out ok. However, developing
that C library was not made easy by some of the design decisions in the
Lua C API.
This was a separate problem, in a different area of the app.
The results were non-intuitive.
In our project, there was some manipulation of very large arrays
(10's to 100's of thousands of entries). At first the inclination
was to do exactly what you suggest -- we imported Torch7, which has
some good array manipulation functions in C, into the system and the
app writers used it. What the ended up writing was (basically)
intermediate_result_1 = torch7.func1(input_array)
intermediate_result_2 = torch7.func2(intermediate_result1)
intermediate_result_3 = torch7.func3(intermediate_result2)
final_result = torch7.func4(intermediate_result3)
where each torch function did some fairly simple
O(N) operation on the array and then generated
a new array.
It turned out to be faster to code the whole thing in a single
Lua loop:
for i=1,n,1 do
r1 = func(array[i])
r2=func2(r1)
r3=func3(r2)
final_result[i] = func4(r3)
end
- one O(N) loop is faster than 4 O(N) loops
- no memory reallocation of new arrays since each
step actually processed each element of the arrays
independently (r1/r2/r3 are numbers, not tables,
in the fragment).
- each of the torch operations, in creating the new array,
basically did an O(2N) operation --- one N was to process
the element, the other was to move it to the new array.
coding the single loop in C rather than Lua would have been the
best, but wasn't an option for various reasons.
Frank Kastenholz