The CSV only has raw data, but the product output has “transactions”. This presents two problems:ġ - The product output is in JSON, the CSV is a CSV.Ģ - The CSV and product output have different structures.
Sometimes in my work, I get CSVs that I need to compare to the product’s output. Time spend programming and practicing data fluency is an investment in ourselves more fluent with data. And, even if there are only a few users, the time is spent programming and practicing data fluency, not mindlessly typing meaningless strings. Although the toy example has only 3 users, this solution works no matter how many test users there are. In practice, don’t store passwords in cleartext : cat test-generator | egrep -o | sort | uniq \
#SEQUEL PRO COPY AS CSV PASSWORD#
I’ll set password to be password in cleartext just for educational purposes. Output: that it’s JSON, we can use jq to make user objects. cat test-generator | egrep -o | sort | uniq \ Output (toy data): now I have structured, newline delimited data. Let’s pull them out: cat test-generator | egrep -o | sort | uniq Since they’re email addresses, I can pull them out using grep. Now that I’m fluent in data, I’ll do it all in one, short pipeline.įirst question, where is the data? It’s in my test file (called test-generator). Before I became fluent in data, I would’ve opened the tests on one monitor, sequel pro on the other, and manually typed in the values for each user. One task I needed to complete was to seed the testing database with my test users. I’m working on a product, and it needs to be tested. If a translator doesn’t exist for your data format, translators to/from JSON are very easy to make in your favorite programming language, as long as your favorite language has libraries for both json and your data format. This structure, with task-dependent JSON magic, can be used to quickly manipulate other data formats like xml, yaml, tsv, dsv, and logfmt. JSON magic (json-table-to-objs | jq 'map(select(.createdDate =. | json-table-to-objs | jq 'map(select(.createdDate =. Here’s an example selecting rows of a CSV where the “createdDate” and “deletedDate” are equal using only json fluency: cat data.csv \ If we’re fluent in JSON, we can leverage that fluency to almost every structured data format by using translators. JSON is the structured data lingua franca.
0 kommentar(er)
