i'd add old app engine logs current bigquery database of logs, they're in odd format.
so started adding (streaming) app engine logs bigquery directly. have 90 days of old logs (from before started adding them bigquery) downloaded using sort of command:
<<path-to>>/appcfg.sh --num_days=0 --severity=0 --include_all request_logs <<war-location>> <<output-file>> which can seen here: https://cloud.google.com/appengine/docs/java/tools/uploadinganapp#command_line_arguments
so downloaded these logs , got text file around 3 gb. in format don't recognize (not csv or json). looks this:
2601:9:6900:b700:c1ca:f9d7:7d6f:9677 - - [27/apr/2015:13:00:39 -0700] "options /mediaserver/<<fileurl>>" 200 109 - "mozilla/5.0 (macintosh; intel mac os x 10_9_5) applewebkit/537.36 (khtml, gecko) chrome/38.0.2125.101 safari/537.36" "<<domain>>" ms=12 cpu_ms=18 cpm_usd=0.000012 instance=00c61b117cd30a702faf40832448daeb1e98 app_engine_release=1.9.19 1:1430164839.664000 [s~<<domain>>/1.383906932052420675].<stdout>: 27 apr 2015 08:00:39, 664 info <<servlet>>.dooptions(): entered : : 1:1430164839.664000 [s~<<domain>>/1.383906932052420675].<stdout>: 27 apr 2015 08:00:39, 664 info <<servlet>>.dooptions(): exiting : : so is, believe, 1 log (of have tens of thousands). first entry standard app engine log, followed number (in case 2, 0 or more) of logs application.
i haven't found online adding these logs bigquery. i'm thinking i'll have parse csv , upload it. if there way directly import bigquery, that'd preferable.
any ideas? thanks.
