Hi and welcome to my site.
This site is broken into 2 parts.
The Tech Journal which is my adventures in technology and is the most active part of the site.
The other half is the Outdoor Journal which is all about my outdoors hobbies.
This main page will always show the most recent post from each half of the site. Click the links above to see more posts from the Tech or Outdoor halves of the site
Looking in said eb-activity.log we have the following line:
in the eb-activity.log
This stack is a node application stack and we didn't write this script. So next step was a P1 AWS support ticket, thankfully we have a level of support that results in fast answers.
AWS confirmed it was an issue with ebnode.py file that 50npm.sh calls.
Specifically:
"Elastic Beanstalk passes STDOUT and STDERR of npm install to Python and Ruby based logger scripts. These loggers expect NPM modules to conform to standards and use UTF-8 characters."
The cause
The root cause here was that some dependency we had in our application contains an NPM module that is using some non UTF-8 charaters and that causes EB to break on a deployment of the new version.
The AWS provided solution
The AWS provided workaround is to modify the ebnode.py script to redirect NPM install output to /dev/null
Specifically look for a lines like
and modify to:
Then the updated ebnode.py would need to be uploaded into S3 (the elasticbeanstalk-$REGION-$ACCOUNTID bucket probably makes the most sense) and using .ebextensions to pull the new ebnode.py file and overwrite the default one.
If your thinking messy and hard, yes and yes.
How i fixed it
I took our whole application and moved it to a docker container. Then I deployed it to AWS Fargate which is the new "run a container without worrying about hosts" service.
I've been informed that AWS is planning on fixing this, however, sounds like it won't be fixed until they bump the node EB environment again.
So if your running a node application in elastic beanstalk be warned, this could happen to you. In our case, it wasn't even a dependency we updated! It was a dependency of a dependency.
I had arranged to meet everyone at the Maple Grove car park, which is just back from the Scenic Railway and normally doesn't have many cars in it. It's also close to the end of the Furber steps which was the planned walk out route.
From the car park we walked along the Prince Henry cliff walk to Echo Point and then onto the start of The Giant Staircase. This was probably a couple of KMs and took us around 30 minutes to complete. The descent down the Giant Staircase was straight forward and done without incident while enjoying some of the great views over the Jamison valley.
The Giant Staircase sounds worse than it really is. Yes it's steep and narrow, but it's not that long. The descent for us took around 20 minutes and that was with a lot of traffic. The climb back up would take longer. I'd allow 45 minutes to an hour. Less if you're really fit!
Once at the bottom of the stairs we started out for the Scenic Railway which is around 3KM from the base of the Giant Staircase.
We stopped for lunch in a beautiful spot at the base of the Katoomba falls. A short distance before the base of the Scenic Railway. We even celebrated the birthday of one of the group members with cake and all! Much to his surprise.
From the base of the Scenic Railway we walked along the new(ish) boardwalk until the dirt track turn off to Ruined Castle, Golden Stairs and Landslide.
The first part of the track before the Landslide is straight forward. However, when the main fork is reached the lower one needs to be taken or you'll end up against the rock face near Malita Point. The track also cut directly through one of the coal seams in the area. The Scenic Railway was in fact built to service the mines down in the valley long before it became the tourist attraction it is now. When the mines closed up, the railway remained and continues to operate to this day. The base of the railway now has lots of exhibits and signs talking about the old mining days. If you've never been down that way before its certainly worth a look.
I also found what looks like an old mine air shaft between the Landslide and the Golden Stairs. I grabbed the torch and had a quick look in the entrance and found the cutting was almost a prefect rectangle continuing dead straight into the rock for as far as I could see. I would love to know more details if anyone out there has some knowledge about what it is and what shafts in the mountain it connects to.
I should add here just in case someone is dumb enough to get silly ideas. Mine shafts are not something that should be explored. They can be highly unstable and a real death trap for the inexperienced.
Next we came to the Landslide, which I have to say isn't the nicest of walking. Its a lot of small loose rocks which is all too easy to slip on. The path over the rock pile is sign posted with arrows and yellow panted stakes in the ground however inexperienced walkers do need to be careful, its easy to miss a few.
Once we were past the landslide its mostly flat walk, past the bottom of the Golden Stairs until the turn off to Ruined Castle. The turn off is signposted with the other track continuing on to Mt Solitary. The track up to Ruined Castle was a bit of a killer! It's got a serious climb that lasts for probably 1KM. Once past this it's mostly flat across the ridge to the castle.
From the bottom of the rocks which make up the Ruined Castle, is a short climb to the top of the rocks for the view.
Ruined Castle itself is well worth the walk, the 360 degree views over the Jamison valley are a real treat.
The walk back was uneventful aside from us all starting to get tired and that Landslide rock pile seeming harder on the way back! Once back at the scenic railway, we walked back up the Furber steps to the car park as we'd missed the last train out. It was starting to get dark around this time (about 6PM) but there was enough light to see us out of the valley, although we did have torches just in case we got caught. We got out of the valley, three very tired and sore walkers!
All up, we covered somewhere between 20 and 24KMs in around 6 hours which I think isn't bad going for one day!
I highly recommend the walk, although you may want to cut the distance down a bit by skipping the Giant Staircase and getting the Scenic Railway down, unless you're after a real workout. However, unless you have at least one experienced leader in the group, I wouldn't recommend the walk as it is not a walk to be taken lightly.
The Blue Mountains website grades the walk as for experienced walkers only and says the following:
"Experienced walkers only - High level of fitness and navigational skill required, minimum 3 in group. Advise friends or police of route and destination times".
I don't really agree with the navigational part unless they class reading signs as that. Either way, treat the walk with respect and make sure you have plenty of water as there isn't any to be found that's safe to drink on the walk itself.
Caveman
This site is broken into 2 parts.
The Tech Journal which is my adventures in technology and is the most active part of the site.
The other half is the Outdoor Journal which is all about my outdoors hobbies.
This main page will always show the most recent post from each half of the site. Click the links above to see more posts from the Tech or Outdoor halves of the site
Most Recent Posts
Posted In: Tech
AWS Elastic Beanstalk deployment failed - invalid byte sequence in UTF-8 - node application
Posted at: 2018-02-27 @ 09:11:52
Recently one of our elastic beanstalk deployments suddenly failed out of the blue with the following error in the EB console.Command failed on instance. Return code: 1 Output: invalid byte sequence in UTF-8. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Looking in said eb-activity.log we have the following line:
in the eb-activity.log
/AppDeployStage0/AppDeployPreHook/50npm.sh] : Activity has unexpected exception, because: invalid byte sequence in UTF-8 (ArgumentError)
This stack is a node application stack and we didn't write this script. So next step was a P1 AWS support ticket, thankfully we have a level of support that results in fast answers.
AWS confirmed it was an issue with ebnode.py file that 50npm.sh calls.
Specifically:
"Elastic Beanstalk passes STDOUT and STDERR of npm install to Python and Ruby based logger scripts. These loggers expect NPM modules to conform to standards and use UTF-8 characters."
The cause
The root cause here was that some dependency we had in our application contains an NPM module that is using some non UTF-8 charaters and that causes EB to break on a deployment of the new version.
The AWS provided solution
The AWS provided workaround is to modify the ebnode.py script to redirect NPM install output to /dev/null
Specifically look for a lines like
check_call([npm_path, '--production', 'install'], cwd=app_path, env=npm_env)
and modify to:
with open(os.devnull, 'w') as devnull:
check_call([npm_path, '--production', 'install'], cwd=app_path, env=npm_env, stdout=devnull)
Then the updated ebnode.py would need to be uploaded into S3 (the elasticbeanstalk-$REGION-$ACCOUNTID bucket probably makes the most sense) and using .ebextensions to pull the new ebnode.py file and overwrite the default one.
If your thinking messy and hard, yes and yes.
How i fixed it
I took our whole application and moved it to a docker container. Then I deployed it to AWS Fargate which is the new "run a container without worrying about hosts" service.
I've been informed that AWS is planning on fixing this, however, sounds like it won't be fixed until they bump the node EB environment again.
So if your running a node application in elastic beanstalk be warned, this could happen to you. In our case, it wasn't even a dependency we updated! It was a dependency of a dependency.
Posted In: Outdoors
Bushwalking - Blue Mountains, out to Ruined Castle from Scenic Railway
Posted at: 2008-09-23 @ 21:44:09
We had wonderful weather for our walk, not a cloud in the sky but also not too hot. Almost prefect walking weather.I had arranged to meet everyone at the Maple Grove car park, which is just back from the Scenic Railway and normally doesn't have many cars in it. It's also close to the end of the Furber steps which was the planned walk out route.
From the car park we walked along the Prince Henry cliff walk to Echo Point and then onto the start of The Giant Staircase. This was probably a couple of KMs and took us around 30 minutes to complete. The descent down the Giant Staircase was straight forward and done without incident while enjoying some of the great views over the Jamison valley.
The Giant Staircase sounds worse than it really is. Yes it's steep and narrow, but it's not that long. The descent for us took around 20 minutes and that was with a lot of traffic. The climb back up would take longer. I'd allow 45 minutes to an hour. Less if you're really fit!
Once at the bottom of the stairs we started out for the Scenic Railway which is around 3KM from the base of the Giant Staircase.
We stopped for lunch in a beautiful spot at the base of the Katoomba falls. A short distance before the base of the Scenic Railway. We even celebrated the birthday of one of the group members with cake and all! Much to his surprise.
From the base of the Scenic Railway we walked along the new(ish) boardwalk until the dirt track turn off to Ruined Castle, Golden Stairs and Landslide.
The first part of the track before the Landslide is straight forward. However, when the main fork is reached the lower one needs to be taken or you'll end up against the rock face near Malita Point. The track also cut directly through one of the coal seams in the area. The Scenic Railway was in fact built to service the mines down in the valley long before it became the tourist attraction it is now. When the mines closed up, the railway remained and continues to operate to this day. The base of the railway now has lots of exhibits and signs talking about the old mining days. If you've never been down that way before its certainly worth a look.
I also found what looks like an old mine air shaft between the Landslide and the Golden Stairs. I grabbed the torch and had a quick look in the entrance and found the cutting was almost a prefect rectangle continuing dead straight into the rock for as far as I could see. I would love to know more details if anyone out there has some knowledge about what it is and what shafts in the mountain it connects to.
I should add here just in case someone is dumb enough to get silly ideas. Mine shafts are not something that should be explored. They can be highly unstable and a real death trap for the inexperienced.
Next we came to the Landslide, which I have to say isn't the nicest of walking. Its a lot of small loose rocks which is all too easy to slip on. The path over the rock pile is sign posted with arrows and yellow panted stakes in the ground however inexperienced walkers do need to be careful, its easy to miss a few.
Once we were past the landslide its mostly flat walk, past the bottom of the Golden Stairs until the turn off to Ruined Castle. The turn off is signposted with the other track continuing on to Mt Solitary. The track up to Ruined Castle was a bit of a killer! It's got a serious climb that lasts for probably 1KM. Once past this it's mostly flat across the ridge to the castle.
From the bottom of the rocks which make up the Ruined Castle, is a short climb to the top of the rocks for the view.
Ruined Castle itself is well worth the walk, the 360 degree views over the Jamison valley are a real treat.
The walk back was uneventful aside from us all starting to get tired and that Landslide rock pile seeming harder on the way back! Once back at the scenic railway, we walked back up the Furber steps to the car park as we'd missed the last train out. It was starting to get dark around this time (about 6PM) but there was enough light to see us out of the valley, although we did have torches just in case we got caught. We got out of the valley, three very tired and sore walkers!
All up, we covered somewhere between 20 and 24KMs in around 6 hours which I think isn't bad going for one day!
I highly recommend the walk, although you may want to cut the distance down a bit by skipping the Giant Staircase and getting the Scenic Railway down, unless you're after a real workout. However, unless you have at least one experienced leader in the group, I wouldn't recommend the walk as it is not a walk to be taken lightly.
The Blue Mountains website grades the walk as for experienced walkers only and says the following:
"Experienced walkers only - High level of fitness and navigational skill required, minimum 3 in group. Advise friends or police of route and destination times".
I don't really agree with the navigational part unless they class reading signs as that. Either way, treat the walk with respect and make sure you have plenty of water as there isn't any to be found that's safe to drink on the walk itself.
Caveman