One Stop Expat Shop

Aleksandra Zajączkowska, Avantika Nair, Emil Braasch

Introduction

As an international student, one of the problems we face when we move to a new country was finding accurate and up-to-date information about the new city in which you now reside. This is the problem we wished to address and seek to solve in this project. Though there are several different places that provide information about more technical tasks like getting your residence permit, Nem-ID and opening a bank account, we wanted to create a website where expatriates can go to get all the information about the city — the best restaurants, fun activities to do & where and how to shop for everyday and luxury items.
In Copenhagen, there are several Facebook groups where a person can find this information if they look through several posts. However, we wanted our website to be a place where people can find all kinds of information about the city sorted into intuitive categories. Thus, we named this platform ‘One Stop Expat Shop’. We initially set out to retrieve all this information from from Facebook groups, Trip Advisor reviews and Twitter pages about Denmark.

2. Challenges

2.1 Cooperation and project work in light of the COVID-19 pandemic

While the One Stop Expat Shop set out to connect expats to a unified platform with information and shared experiences, we ourselves shortly before commencing the project on March 18th experienced the effects of the COVID-19 pandemic, in particular, due to social restrictions put upon everyone under the lockdown of Denmark. While everything at first seemed disconnected, a quick adoption of Zoom and Microsoft Teams for meetings and virtual knowledge sharing beyond Slack enabled us to stay on track with our project. We sought to do weekly meetings and follow-ups, at times more or less successful. While the support of virtual platforms enabled us to continue the project, the disconnection of social interaction throughout the project work phase at times made it difficult to discuss topics, decide upon changes, plan, describe and work on tasks.

2.2 Selection of data

A challenge to our project was our selection of data type. The nature of the project reflects a wish to provide qualitative information and experiences shared from other expats. While the data can vary in type, form and content, it is solely qualitative and not quantitative. This means that data sets had to be obtained from already existing providers of expat experiences, such as forums, web blogs, social media feeds or news article publishers. The process of finding and deciding upon a data set was lengthy and had us occupied during the first half of the project period. An option was to make up data, but was deemed too demanding and would slow our project process even further. Up until shortly before the interim pitch on May 6th we were discussing other options such as change of scope and project theme to more easily access fitting data sets. An example was to change the project to a platform for tracking the changes in COVID-19 cases, but we ended up sticking to the original plan and attempted our best at the datasets available.

Methodology

3.1 Internal Organisation

As a team, we adopted an agile way of working that complemented our goal to create a website. We used the Kanban-style application Trello to manage ongoing and future tasks among team members. We had weekly meetings where we reviewed the tasks and discussed and set priorities. Using this application, helped us work better as a team as it led to less confusion on who owns which task and increased the team’s ability to focus on tasks.

3.2. Getting the Data

Getting the data was a challenge from the beginning. At the beginning, we were considering scraping Facebook groups or TripAdvisor. However, it turned out to be out of the scope of this project. Such advanced websites do not create blocking algorithms to not allow scraping. Then our idea was to find Twitter accounts with related content and get the data through APIs. However, this plan despite being relatively easy to complete had its obstacles. Due to Covid19 crisis, approval to get a developer account took 3 weeks. The idea was abandoned and instead of this, we decided to scrape the data from yourdanishlife.dk website.

Scraping was done in Python programming language with the help of BeautifulSoup library. To be able to get relevant information for expats, we have found a website which gathered articles. We created a code which iterated through all articles and took from them all relevant information which was the websites’ category, the title of the article and full body text. To be able to write a code we had to go through relevant divs and went through the HTML structure of the website. Due to the fact that on the main page there were only previews of articles the code had to “find” all divs, and then later connect the relevant class in them which stored links to each article. This method allowed the loop to iterate through all previews in an article, go inside the link and get relevant information from the inside of the link which was the category, title and text. The text was parsed and then appended in the form of DataFrame. At the end, the data frame was transformed into a csv file.

This allowed us to access many articles, becoming the base for our project

3.3 Data cleaning

The next step was to clean the data. Due to the fact that Python saves scrapped files in a slightly different format than Excel is able to read, it required a bit of transformation. We transformed files through ANSI encoding and thanks to that, Excel was able to read all variables in the articles and did not show distorted variables. Due to the fact that data was not big, we decided to clean it in Excel. Thanks to “Data” Excel functionality, we were able to fix csv biases and relevant content was divided into columns. After accessing the data, we realized that the categories which are scraped from the website did not fit to our needs. Thus, we have come up with our own structure, including five main categories relevant for expats:

• Wine & Dine
• Culture
• To do
• Places
• Daily life

To be able to assign desired categories to articles the function find() and replace() was used. First, we used key phrases to distinguish desired articles — what they contain i.e. “drink”, “wine”, “eat”,”dinner” to recognize the “Wine & Dine” category. Then a dummy variable was assigned to these articles. Based on dummy variable recognition we were able to replace existing categories with our own. The same way was used to change and rename the rest of the categories.

3.4 UI

When we considered which functions to include in the website, we decided to focus on the search function and the categories a visitor can find information on. At this stage, we wanted to eliminate all possible noise in terms of additional non-essential functions. As you can see from the screenshot, the search function appears in the front and centre of the page. The name of the website and the city appears clearly so that users are aware of which location they are discovering. The search function has the text ‘Search’ & a search icon which would notify the user about the search as well as a text box for them to type what they want to get information about.

Below the search function, the various categories available can be seen with the star icon. At the current stage, we have decided to include the five categories — Wine & Dine, Culture, To-do, Places and Daily life. This is because we found that most of what an expat asks about on Facebook groups and blogs fall under these five categories.

When a user types a word or phrase into the search bar and clicks enter, the most relevant articles will appear below the search icon. As it can be seen from the screenshot below, when we search for the words ‘best wine’, we can see the result of the search as ‘Copenhagen’s coziest wine bar’.

Apart from searching for specific information, a visitor can also choose from the list of categories available. For example, if ‘Wine & Dine’ is chosen, we see the various articles about it below the search and the title of the page is changed to the category. This was done in a hope to reduce confusion for the end-user about which page they are currently on. To return to the main page, the user must click the button on the top left side of the page. We used standard design features so it would be easy for a user to use the page.

For the UI purposes, we decided to make use of the Figma prototyping tool. The use was very convenient as Figma allows for low fidelity prototypes creation. Thanks to its functionalities, creation of a click-through prototype was possible. Figma allows for full freedom with design and functionalities possibilities. It enabled us to design the website in a way that searching is easy. It also gave us an idea of how we can structure the project further. Figma was also chosen because it allows in a later stage to copy the essential code, so the design can be “moved” to final version of a website. It was chosen to let us put our imagination into action and try to develop different functionalities in a form of prototype. Thus, we were able to add more and more functions to simple product and think how we can connect the data with UI.

3.5 Search Functionality

Click-through prototype was a great idea because it gave us inspiration how can we use our cleaned and gathered data sample. After brainstorming session, it was decided that search functionality will be implemented in Java Script as it will allow the data to be displayed in html website form. Also, by connecting it to html, we knew that as a further step, it will be possible to move our Figma design into final prototype. Since our data was in csv file, it was very easy for us to transform it into JS array. We created an array which consisted of essential information such as category, title, text, ID of category. Then the class was created to store our array. The class consisted of functions inside, which were responsible for displaying relevant information. The loop was iterating through empty string. It was made to enable connection between our data (array) and html. Through getElementbyId and display.innerHTML, we were able to present our articles.

Following the prototype, the page with Categories was created as well as page with all articles together. The result is the following:

By creating Figma prototype, we realized that search function should be displayed in both views and it should enable to search through keywords. Thus, the following function was created:

First, search function changed all typed signs into the lower case. Due to the fact that our data was interconnected with html, the created search function also searched through divs which was represented by categories and titles. Thanks to that search function could “catch” even unfinished phrases and words. It displayed the approximate match allowing for the most relevant choices to be displayed. It was flexible in the mean that it allowed for few similar results to be displayed if the key phrase matched. Thanks to that user could have bigger choice. The following picture shows one result displayed:

Also, due to the fact that the function innerHTML allowed for fluent display of our articles. Also, the Categories page was clickable. When entered, it displayed certain Article which was matched to desired category. It was possible because by creating class of articles, we added ID of category. Then, following this method (the same go through divs, connect by innerHTML loop), the class of categories was made and interconnected two functions by the same ID. This time the class of categories had ID connected and named in the same way as in the case of articles. The result was the following:

Clicking in Culture Shows as all results matched to culture, in this case i.e. Danish Flag.

Conclusion and Learning

After working on this project for the duration of this learning programme, we learnt that it is essential to scope for the data required and access to that data in the initial stage. It is important to be flexible about the data and try to solve the problem given the limited resources and other hurdles. One major hurdle was that we were not able to work together in person due to the Covid-19 pandemic. As the time passed, we were able to find our rhythm as a team but this took longer than we predicted. Also, we believe that creating sample small elements and adding more on top of that helped us to understand how agile is done in practise. We decided i.e. to create website functionality based only on few samples from our dataset, to try out our possibilities. When it worked, we could add more.

Further steps

The website we have must be improved on by testing the prototype with various audiences and increasing the amount of information available on the website. As a part of further steps, we could enrich our data set by including Twitter API data. Since we at last were granted a Twitter developer account, it could be made possible. By using that, the text could be cleaned and assigned to our categories. This time it would not be in the form of articles but in the form of an “advice” due to the fact that Twitter allows only for short posts of 280 characters.

In addition to this, we would like to add a recommendation system, where users can rate various articles based on how useful they find it. After these functions have been implemented, we would like to look into allowing users to log in and write short opinion articles and submit them for the website to use.

Everything can be found under this GitHub Repo.

The link to prototype can be found here.

Authors:

Aleksandra Zajączkowska LinkedIn

Avantika Nair LinkedIn

Emil Braasch LinkedIn