Code Sample, a copy-pastable example if possible

See https://travis-ci.org/MacPython/pandas-wheels/jobs/448674095 for some specific test failures. The basic setup on a 32bit system is along the lines of:

In [2]: left = np.arange(5)
In [3]: tree = pd._libs.interval.IntervalTree(left, left+2, leaf_size=2)
---------------------------------------------------------------------------
TypeError: Cannot cast array data from dtype('int64') to dtype('int32') according to the rule 'safe'

xref #23353 (comment)

Problem description

IntervalTree construction fails for 32bit when n_elements > leaf_size.

Expected Output

I'd expect construction to be successful.