[C++] 32 bit MSVC doesn't build

[asfimport]

When specifying Win32 as a platform, and building with MSVC, the build fails with the following compile errors :

C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(70,59): error C3861: '__popcnt64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj]
C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(204,7): error C3861: '_BitScanReverse64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj]
C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(250,7): error C3861: '_BitScanForward64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj] 

Environment: Win32, MSVC
Reporter: Arkadiy Vertleyb

PRs and other links:

• GitHub Pull Request #13376
• GitHub Pull Request #13519
• GitHub Pull Request #13532

_{Note: This issue was originally created as ARROW-16778. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
[~avertleyb] Would you like to submit a Github PR to fix those issues?

[asfimport]

Arkadiy Vertleyb:
I was going to, but after I got the build fixed the arrow-utility-test hangs. Not sure if it is related to my fix, haven't figured out yet how debug it properly.

This was my patch:

index 8583e10b2..5122cb3d5 100644
— a/cpp/src/arrow/util/bit_util.h
+++ b/cpp/src/arrow/util/bit_util.h
@@ -67,7 +67,14 @@ static constexpr uint8_t kBytePopcount[] = {
5, 4, 5, 5, 6, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 3, 4, 4, 5, 4, 5, 5, 6,
4, 5, 5, 6, 5, 6, 6, 7, 4, 5, 5, 6, 5, 6, 6, 7, 5, 6, 6, 7, 6, 7, 7, 8};

-static inline uint64_t PopCount(uint64_t bitmap) { return ARROW_POPCOUNT64(bitmap); }
+static inline uint64_t PopCount(uint64_t bitmap) {
+#if defined(_MSC_VER) &&  && )defined(_M_X64)

• const uint32_t* p = reinterpret_cast<const uint32_t*>(&bitmap);
• return ARROW_POPCOUNT32(p) + ARROW_POPCOUNT32((p + 1));
  +#else
• return ARROW_POPCOUNT64(bitmap);
  +#endif
  +}
  static inline uint32_t PopCount(uint32_t bitmap) { return ARROW_POPCOUNT32(bitmap); }

//
@@ -199,7 +206,7 @@ static inline int CountLeadingZeros(uint64_t value) {
#if defined(clang) || defined(GNUC)
if (value == 0) return 64;
return static_cast(__builtin_clzll(value));
-#elif defined(_MSC_VER)
+#elif defined(_MSC_VER) && (defined(_M_AMD64) || defined(_M_X64))
unsigned long index; // NOLINT
if (_BitScanReverse64(&index, value)) { // NOLINT
return 63 - static_cast(index);
@@ -245,7 +252,7 @@ static inline int CountTrailingZeros(uint64_t value) {
#if defined(clang) || defined(GNUC)
if (value == 0) return 64;
return static_cast(__builtin_ctzll(value));
-#elif defined(_MSC_VER)
+#elif defined(_MSC_VER) && (defined(_M_AMD64) || defined(_M_X64))
unsigned long index; // NOLINT
if (_BitScanForward64(&index, value)) {
return static_cast(index);

[asfimport]

Antoine Pitrou / @pitrou:
Under Linux you would debug using gdb <binary-dir>/arrow-utility-test. Under Windows I have no idea.

[asfimport]

Arkadiy Vertleyb:
Can't build Debug version. Is:

cmake --build . --config Debug

The right way to do it?

Getting the following:

Performing copy step for 'googletest_ep'
Error copying file "C:/.../arrow/cpp/build32_debug/googletest_ep-prefix/bin/gtest.dll" to "C:/.../arrow/cpp/build32_debug/release//RELEASE".
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'C:...\arrow\cpp\build32_debug\CMakeFiles\d9834a26c074484d22
c6b56466b8c71e\googletest_ep-copy.rule;C:...\arrow\cpp\build32_debug\CMakeFiles\0e7b907092777ef17a2918749093be30\googletest_ep-complete.rule;C:...\arrow\cpp\build32_debug\CMakeFiles\4d50031ad3e0595d6aa1
aebb9c9d11b8\googletest_ep.rule' exited with code 1. [C:...\arrow\cpp\build32_debug\googletest_ep.vcxproj]

[asfimport]

Antoine Pitrou / @pitrou:
You must first configure the CMake build type.
For this you can pass the -DCMAKE_BUILD_TYPE=Debug flag in the CMake configuration command.

[asfimport]

Will Jones / @wjones127:
Also there are some other useful debug settings. See the base-debug preset in CMakePresets.json.

[asfimport]

Will Jones / @wjones127:
FYI I'm working on creating a win32 build right now to see if I can reproduce what you are seeing.

[asfimport]

Arkadiy Vertleyb:
Thanks Will.
Saw you latest comment at Bloomberg.
Please also keep in mind that using -DCMAKE_BUILD_TYPE=Debug somehow ends up with treating warnings as errors, which doesn't work well in a 32 bit system because of conflicts between int64_t and size_t. As I understand this comes from Google guidelines conflicting with C++ standard. Not sure how to avoid this.
As you probably guessed by now, I am no cmake expert :-(

[asfimport]

Will Jones / @wjones127:
Yes I got that error as warning too. For now I was able to bypass with -DARROW_CXXFLAGS='/wd4244 /wd4554'.

[asfimport]

Will Jones / @wjones127:
Also note, it seems that our bundled version of Boost Build doesn't compile on VS2022. We may wish to upgrade that.

I'll go forward in VS2019 for now.

[asfimport]

Arkadiy Vertleyb:
2019 is good for now.

[asfimport]

Will Jones / @wjones127:
[~avertleyb] I've been able to get both of the following configurations to compile. Do you want to try them out and lmk if they work for you?

@REM Build using Ninja
cmake.EXE ^
 -DARROW_DEPENDENCY_SOURCE=BUNDLED ^
 -DCMAKE_BUILD_TYPE=Debug ^
 -DARROW_BUILD_TESTS=ON ^
 "-DARROW_CXXFLAGS=/wd4244 /wd4554 /wd4018" ^
 -DARROW_BUILD_INTEGRATION=OFF ^
 -DARROW_EXTRA_ERROR_CONTEXT=ON ^
 -DARROW_BUILD_STATIC=OFF ^
 -DARROW_WITH_RE2=OFF ^
 -DARROW_WITH_UTF8PROC=OFF ^
 -DCMAKE_INSTALL_PREFIX=c:/Users/voltron/arrow/cpp/build/user-cpp-debug-win32-alt/dist ^
 -S%USERPROFILE%/arrow/cpp ^
 -B%USERPROFILE%/arrow/cpp/build/user-cpp-debug-win32 ^
 -G Ninja

@REM Or build using Visual Studio
cmake.EXE ^
 -DARROW_DEPENDENCY_SOURCE=BUNDLED ^
 -DCMAKE_BUILD_TYPE=Debug ^
 -DARROW_BUILD_TESTS=ON ^
 "-DARROW_CXXFLAGS=/wd4244 /wd4554 /wd4018" ^
 -DARROW_BUILD_INTEGRATION=OFF ^
 -DARROW_EXTRA_ERROR_CONTEXT=ON ^
 -DARROW_BUILD_STATIC=OFF ^
 -DARROW_WITH_RE2=OFF ^
 -DARROW_WITH_UTF8PROC=OFF ^
 -DCMAKE_INSTALL_PREFIX=c:/Users/voltron/arrow/cpp/build/user-cpp-debug-win32-alt/dist ^
 -S%USERPROFILE%/arrow/cpp ^
 -B%USERPROFILE%/arrow/cpp/build/user-cpp-debug-win32-alt ^
 -G "Visual Studio 16 2019" ^
 -A Win32
 

[asfimport]

Antoine Pitrou / @pitrou:
Instead of disabling warnings one-by-one, you can change the BUILD_WARNING_LEVEL CMake variable. I don't know why we are not documenting it ( @kou do you know?), but it is described thusly in cmake_modules/SetupCxxFlags.cmake:

# BUILD_WARNING_LEVEL add warning/error compiler flags. The possible values are
# - PRODUCTION: Build with `-Wall` but do not add `-Werror`, so warnings do not
#   halt the build.
# - CHECKIN: Build with `-Wall` and `-Wextra`.  Also, add `-Werror` in debug mode
#   so that any important warnings fail the build.
# - EVERYTHING: Like `CHECKIN`, but possible extra flags depending on the
#               compiler, including `-Wextra`, `-Weverything`, `-pedantic`.
#               This is the most aggressive warning level.

# Defaults BUILD_WARNING_LEVEL to `CHECKIN`, unless CMAKE_BUILD_TYPE is
# `RELEASE`, then it will default to `PRODUCTION`. The goal of defaulting to
# `CHECKIN` is to avoid friction with long response time from CI.

[asfimport]

Will Jones / @wjones127:
Wow that will save me a lot of time! :)

[asfimport]

Arkadiy Vertleyb:
@wjones127 This builds with three path parameters removed to use defaults and with my patch applied.

But then ctest crashes running arrow-io-file-test.