zachlankton
Aug 13, 2022

Hello Everyone, I am trying to set up a development environment to start working with the code base and I'm having a bit of trouble. I am trying to use the devcontainer that is included in this repo.

Here are the things I've tried:

Current DevContainer Config not working.

Tried Using Elixir 1.10 (Current Setting):

Running make yields these results:

==> config (compile)
ERROR: OTP release 22 does not match required regex 23|24|25
ERROR: compile failed while processing /workspaces/couchdb/src/config: rebar_abort
make: *** [Makefile:123: couch] Error 1

Tried Using Elixir 1.11

It would compile, but would not pass all tests (make check):

** (Mix) You're trying to run :couchdbtest on Elixir v1.11.4 but it has declared in its mix.exs file it supports only Elixir ~> 1.13

Tried Using 1.13 - Failed Docker Build:

#0 2.575 Reading package lists...
#0 2.933 + apt-get install -y --no-install-recommends libmozjs-60-dev libicu-dev
 python3-venv python3-pip python3-sphinx nodejs
#0 2.940 Reading package lists...
#0 3.297 Building dependency tree...
#0 3.384 Reading state information...
#0 3.437 E: Unable to locate package libmozjs-60-dev

Changed Dockerfile

-ENV SM_VSN=60
+ENV SM_VSN=78

and while I was at at it I changed the buster node target to bullseye:

- echo "deb https://deb.nodesource.com/node_10.x buster main" | tee /etc/apt/sources.list.d/nodesource.list; \
- echo "deb-src https://deb.nodesource.com/node_10.x buster main" | tee -a /etc/apt/sources.list.d/nodesource.list
+ echo "deb https://deb.nodesource.com/node_10.x bullseye main" | tee /etc/apt/sources.list.d/nodesource.list; \
+ echo "deb-src https://deb.nodesource.com/node_10.x bullseye main" | tee -a /etc/apt/sources.list.d/nodesource.list

Re ran the container and... more problems... now the ./dev/run is failing:

[ * ] Setup environment ... ok
[ * ] Ensure CouchDB is built ... ok
[ * ] Ensure Erlang boot script exists ... ok
[ * ] Prepare configuration files ... ok
[ * ] Start node node1 ... ok
[ * ] Start node node2 ... ok
[ * ] Start node node3 ... ok
[ * ] Check node at http://127.0.0.1:15984/ ... failed: <urlopen error [Errno 111] Connection refused>
[ * ] Check node at http://127.0.0.1:25984/ ... failed: <urlopen error [Errno 111] Connection refused>
[ * ] Check node at http://127.0.0.1:35984/ ... failed: <urlopen error [Errno 111] Connection refused>
...

found some logs in /dev/logs/node[1-3].log --->

{"init terminating in do_boot",{load_failed,[supervisor,proc_lib,logger_backend,logger_simple_h,lists,logger,logger_config,logger_filters,gen_event,kernel,logger_server,heart,gen_server,gen,file_io_server,file_server,erl_eval,ets,file,error_logger,filename,erl_parse,erl_lint,code_server,application_master,code,application,application_controller,error_handler]}}
init terminating in do_boot ({load_failed,[supervisor,proc_lib,logger_backend,logger_simple_h,lists,logger,logger_config,logger_filters,gen_event,kernel,logger_server,heart,gen_server,gen,file_io_server,file_server,erl_eval,ets,file,error_logger,filename,erl_parse,erl_lint,code_server,application_master,code,application,application_controller,error_handler]})
Crash dump is being written to: erl_crash.dump...done

looking at the erl_crash.dump file shows a lot of stuff! Over 24,000 lines!!!

Could not find anything helpful...

Where I'm at now:

I rewrote the Dockerfile to use

Erlang 23
Elixir 1.13
Spidermonkey 60

FROM erlang:23
# elixir expects utf8.
ENV ELIXIR_VERSION="v1.13.4" \
	LANG=C.UTF-8
RUN set -xe \
	&& ELIXIR_DOWNLOAD_URL="https://github.com/elixir-lang/elixir/archive/${ELIXIR_VERSION}.tar.gz" \
	&& ELIXIR_DOWNLOAD_SHA256="95daf2dd3052e6ca7d4d849457eaaba09de52d65ca38d6933c65bc1cdf6b8579" \
	&& curl -fSL -o elixir-src.tar.gz $ELIXIR_DOWNLOAD_URL \
	&& echo "$ELIXIR_DOWNLOAD_SHA256 elixir-src.tar.gz" | sha256sum -c - \
	&& mkdir -p /usr/local/src/elixir \
	&& tar -xzC /usr/local/src/elixir --strip-components=1 -f elixir-src.tar.gz \
	&& rm elixir-src.tar.gz \
	&& cd /usr/local/src/elixir \
	&& make install clean \
	&& find /usr/local/src/elixir/ -type f -not -regex "/usr/local/src/elixir/lib/[^\/]*/lib.*" -exec rm -rf {} + \
	&& find /usr/local/src/elixir/ -type d -depth -empty -delete
# Install SpiderMonkey 60 and tell CouchDB to use it in configure
ENV SM_VSN=60
# Use NodeSource binaries for Node.js (Fauxton dependency)
RUN set -ex; \
 curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -; \
 echo "deb https://deb.nodesource.com/node_10.x buster main" | tee /etc/apt/sources.list.d/nodesource.list; \
 echo "deb-src https://deb.nodesource.com/node_10.x buster main" | tee -a /etc/apt/sources.list.d/nodesource.list
RUN set -ex; \
 apt-get update; \
 apt-get install -y --no-install-recommends \
 libmozjs-${SM_VSN}-dev \
 libicu-dev \
 python3-venv \
 python3-pip \
 python3-sphinx \
 nodejs
# Documentation theme
RUN pip3 install sphinx_rtd_theme

Now I can run ./configure && make as well as ./dev/run

but I have 2 failing tests and I have not started making changes yet... this is a fresh clone from main

=======================================================
 Failed: 2. Skipped: 0. Passed: 1297.
ERROR: One or more eunit tests failed.
ERROR: eunit failed while processing /workspaces/couchdb/src/couch: rebar_abort
make: *** [Makefile:175: eunit] Error 1

If I scroll up I can find the 2 failed tests:

Auto compactions triggered to update collators
 couchdb_views_tests:476: view_collator_auto_upgrade_on_open...*failed*
in function couchdb_views_tests:read_header/1 (test/eunit/couchdb_views_tests.erl, line 1047)
in call from couchdb_views_tests:'-view_collator_auto_upgrade_on_open/1-fun-4-'/2 (test/eunit/couchdb_views_tests.erl, line 494)
in call from eunit_test:run_testfun/1 (eunit_test.erl, line 71)
in call from eunit_proc:run_test/1 (eunit_proc.erl, line 522)
in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 347)
in call from eunit_proc:handle_test/2 (eunit_proc.erl, line 505)
in call from eunit_proc:tests_inorder/3 (eunit_proc.erl, line 447)
in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 337)
**error:{badmatch,{error,enoent}}
 output:<<"">>

 couchdb_views_tests:503: view_collator_auto_upgrade_on_update...*failed*
in function couchdb_views_tests:read_header/1 (test/eunit/couchdb_views_tests.erl, line 1047)
in call from couchdb_views_tests:'-view_collator_auto_upgrade_on_update/1-fun-4-'/2 (test/eunit/couchdb_views_tests.erl, line 533)
in call from eunit_test:run_testfun/1 (eunit_test.erl, line 71)
in call from eunit_proc:run_test/1 (eunit_proc.erl, line 522)
in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 347)
in call from eunit_proc:handle_test/2 (eunit_proc.erl, line 505)
in call from eunit_proc:tests_inorder/3 (eunit_proc.erl, line 447)
in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 337)
**error:{badmatch,{error,enoent}}
 output:<<"">>

 couchdb_views_tests:539: view_collator_auto_upgrade_can_be_disabled...[0.141 s] ok
 [done in 11.964 s]
 [done in 18.704 s]

As I understand, any contributions that I make will need to pass ALL tests, but it appears my fresh clone of main is failing these 2 tests.

What can I do to correct this problem?

Should I be using a different branch?

Or is there something that I'm missing when trying to set up my development environment?

Thanks in advance!

Answered by zachlankton

Aug 15, 2022

It actually appears that there is a deeper issue with VS Code Dev Containers (maybe just in windows?), and it has something to do with the way the folder is mounted in the dev container.

I've tried multiple container configurations and docker images and nothing I do can get all the tests to pass using a devcontainer. The same 2 tests fail every time.

However... when I clone the repo inside the container and run all the config, make, dev/run, make/check commands everything works perfectly! All tests pass!

Final Working Solution (what worked for me)

Use Docker to run an empty CouchDB CI Container. These images have all the dependencies needed to run and pass all the tests. Additionally, th...

View full answer

Replies: 2 comments 1 reply

zachlankton
Aug 15, 2022
Author

It actually appears that there is a deeper issue with VS Code Dev Containers (maybe just in windows?), and it has something to do with the way the folder is mounted in the dev container.

I've tried multiple container configurations and docker images and nothing I do can get all the tests to pass using a devcontainer. The same 2 tests fail every time.

However... when I clone the repo inside the container and run all the config, make, dev/run, make/check commands everything works perfectly! All tests pass!

Final Working Solution (what worked for me)

Use Docker to run an empty CouchDB CI Container. These images have all the dependencies needed to run and pass all the tests. Additionally, they work great for a development environment.

docker run -d \
 --name couchdb-dev \
 --env SM_VSN=78 \
 --user=root \
 apache/couchdbci-debian:bullseye-erlang-25.0.2 \
 sleep infinity

Then attach to it using VS Code. (May need to install VS Code Remote Extension )

image

Open A Folder to Work in...

image

and then open a new terminal and enter the following commands:

# clone the repo (replace with your fork if needed)
git clone https://github.com/apache/couchdb
cd couchdb
./configure && make
# Make sure dev run works, open url, login, make db, delete db, etc...
./dev/run
# Make sure all tests are passing
make check
# Time to Relax!

Closing Thoughts

This is not the first time I have been burned by some quirk in VS Code Dev Containers (in windows). I use them sometimes, but most of the time I end up doing some variation of the above: Either spinning up my own container and attaching to it or spinning up an entire Linux VM and using VS Code Remote SSH extension.

I swear I am so close to scrapping Windows finally and fully committing to a Linux Desktop.

0 replies

Answer selected by zachlankton

big-r81
Aug 15, 2022
Collaborator

Hi Zach,

thank you for taking a deeper dive into this. I never used the devcontainer. If there is anything that needs to be fixed to help others, please open a PR!

For building (and releasing binaries) for windows, you can use our glazier. But this will install a lot other dependencies and you should only use it in a vm. IIRC some tests failed in windows, but I'm not sure if the problems was the os...

As you mentioned, it might be easier to use a empty CouchDB CI Container or a simple linux vm for developing.
But a consistent and simple dev environment is on the todo list...

1 reply

@zachlankton

zachlankton Aug 15, 2022
Author

Thanks Ronny,

I think I can help out with this!

I will work on a PR to update the existing devcontainer and documentation.

I'll also try spinning up a Linux VM Desktop to see if the same problem exists there.

I don't currently have a mac to test this on, maybe someone in the community can give that a try when I get the devcontainer config updated?

Help Setting Up Development Environment using .devcontainer #4145

Uh oh!

zachlankton Aug 13, 2022

Current DevContainer Config not working.

Tried Using Elixir 1.10 (Current Setting):

Tried Using Elixir 1.11

Tried Using 1.13 - Failed Docker Build:

Where I'm at now:

Final Working Solution (what worked for me)

Replies: 2 comments · 1 reply

Uh oh!

Uh oh!

zachlankton Aug 15, 2022 Author

Final Working Solution (what worked for me)

Closing Thoughts

Uh oh!

big-r81 Aug 15, 2022 Collaborator

Uh oh!

Uh oh!

zachlankton Aug 15, 2022 Author

zachlankton
Aug 13, 2022

Replies: 2 comments 1 reply

zachlankton
Aug 15, 2022
Author

big-r81
Aug 15, 2022
Collaborator

zachlankton Aug 15, 2022
Author